{"record":{"id":"1df3f127e4d1d83a","repo":"stanfordnlp/CoreNLP","slug":"s-bare-tagged-word-being-wrapped-in-frag-s","errorCode":null,"errorMessage":"%s: Bare tagged word being wrapped in FRAG %s","messagePattern":"(.+?): Bare tagged word being wrapped in FRAG (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"info","filePath":"src/edu/stanford/nlp/trees/international/arabic/ArabicTreeNormalizer.java","lineNumber":217,"sourceCode":"          prd.label().setValue(super.normalizeNonterminal(prd.label().value()));\n        }\n      }\n    }\n\n    //Mark *only* subjects in verb-initial clauses\n    if(retainNPSbj) {\n      TregexMatcher m = npSbjPattern.matcher(tree);\n      while (m.find()) {\n        Tree match = m.getMatch();\n        match.label().setValue(\"NP\");\n      }\n    }\n\n    if (tree.isPreTerminal()) {\n      // The whole tree is a bare tag: bad!\n      String val = tree.label().value();\n      if (val.equals(\"CC\") || val.startsWith(\"PUNC\") || val.equals(\"CONJ\")) {\n        log.warn(String.format(\"%s: Bare tagged word being wrapped in FRAG %s\", this.getClass().getName(),tree.pennString()));\n        tree = tf.newTreeNode(\"FRAG\", Collections.singletonList(tree));\n      } else {\n        log.warn(String.format(\"%s: Bare tagged word %s\", this.getClass().getName(), tree.pennString()));\n      }\n    }\n\n    //Add start symbol so that the root has only one sub-state. Escape any enclosing brackets.\n    //If the \"tree\" consists entirely of enclosing brackets e.g. ((())) then this method\n    //will return null. In this case, readers e.g. PennTreeReader will try to read the next tree.\n    while (tree != null && (tree.value() == null || tree.value().isEmpty()) && tree.numChildren() <= 1) {\n      tree = tree.firstChild();\n    }\n\n    if (tree != null && !tree.value().equals(rootLabel)) {\n      tree = tf.newTreeNode(rootLabel, Collections.singletonList(tree));\n    }\n\n    return tree;","sourceCodeStart":199,"sourceCodeEnd":235,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/trees/international/arabic/ArabicTreeNormalizer.java#L199-L235","documentation":"A warning from ArabicTreeNormalizer.normalizeWholeTree: if the entire input tree is a single preterminal (a bare tagged word, e.g. '(PUNC .)'), that is 'bad' structure, so for coordination tags (CC), punctuation (PUNC*), or CONJ the normalizer wraps it in a synthetic FRAG node and logs '<class>: Bare tagged word being wrapped in FRAG <pennString>'. Other tags are logged as 'Bare tagged word' without wrapping (a separate message).","triggerScenarios":"Calling normalizeWholeTree on a tree consisting of exactly one preterminal — typically when sentence segmentation fed a single token (a lone period, conjunction, or particle) to the parser/normalizer.","commonSituations":"Tokenizing/splitting text that yields one-word 'sentences' (standalone punctuation, 'و' conjunction); parsing sentence fragments from Arabic text then normalizing them for training data.","solutions":["Avoid sending single-token trees to the parser/normalizer by filtering one-token segments before parsing","Accept the FRAG wrapping: output tree remains well-formed for downstream use","If other bare tags are needed, pre-wrap them yourself in a suitable node before normalization","Fix upstream sentence splitting so lone punctuation stays attached to adjacent sentences"],"exampleFix":"// before\nTree t = parser.apply(word); // word == \".\"  -> whole tree is bare preterminal\nnormalizer.normalizeWholeTree(t, tf); // warns, wraps in FRAG\n// after\nif (word.matches(\"[.?!،؛]+\")) return; // skip lone punctuation segments\ntree = normalizer.normalizeWholeTree(parser.apply(word), tf);","handlingStrategy":"validation","validationCode":"// skip single-preterminal trees before parsing/normalizing\nif (text.split(\"\\\\s+\").length <= 1 && text.matches(\"[.?!،؛]?\")) {\n  return; // or merge with the neighboring sentence\n}","typeGuard":null,"tryCatchPattern":"try {\n  Tree out = normalizer.normalizeWholeTree(tree, tf);\n  if (out.isPreTerminal()) log.warn(\"Tree is still a bare preterminal after normalization\");\n} catch (Exception e) {\n  log.warn(\"Normalization failed\", e);\n}","preventionTips":["Improve sentence splitting so lone punctuation/conjunctions never form their own segment","Post-check that parser output trees are not single preterminals","For one-word fragments of content words, wrap in (FRAG ...) yourself before processing"],"tags":["java","nlp","trees","arabic","sentence-segmentation"],"backgroundTag":"unexpected-response-shape","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}