{"record":{"id":"4d14ba4d562fb41f","repo":"stanfordnlp/CoreNLP","slug":"s-missing-tag-for-s","errorCode":null,"errorMessage":"%s: missing tag for %s","messagePattern":"(.+?): missing tag for (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/edu/stanford/nlp/trees/international/arabic/ArabicTreeNormalizer.java","lineNumber":162,"sourceCode":"            } else {\n              // TODO(spenceg): Does this help?\n              String newLemma = lexMapper.map(null, lemma);\n              if (newLemma == null || newLemma.trim().isEmpty()) {\n                newLemma = lemma;\n              }\n              String newMorphAnalysis = newLemma + MorphoFeatureSpecification.LEMMA_MARK + morphAnalysis;\n              cl.setOriginalText(newMorphAnalysis.intern());\n            }\n\n          } else {\n            log.error(String.format(\"%s: Cannot store morph analysis in non-CoreLabel: %s\",this.getClass().getName(),t.label().getClass().getName()));\n          }\n        }\n\n      } else if (t.isPreTerminal()) {\n\n        if (t.value() == null || t.value().isEmpty()) {\n          log.warn(String.format(\"%s: missing tag for %s\",this.getClass().getName(),t.pennString()));\n        } else if(t.label() instanceof HasTag) {\n          ((HasTag) t.label()).setTag(t.value());\n        }\n\n      } else { //Phrasal nodes\n\n        // there are some nodes \"/\" missing preterminals.  We'll splice in a tag for these.\n        int nk = t.numChildren();\n        List<Tree> newKids = new ArrayList<>(nk);\n        for (int j = 0; j < nk; j++) {\n          Tree child = t.getChild(j);\n          if (child.isLeaf()) {\n            log.warn(String.format(\"%s: Splicing in DUMMYTAG for %s\",this.getClass().getName(),t.toString()));\n            newKids.add(tf.newTreeNode(\"DUMMYTAG\", Collections.singletonList(child)));\n\n          } else {\n            newKids.add(child);\n          }","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/trees/international/arabic/ArabicTreeNormalizer.java#L144-L180","documentation":"A warning (not exception) from ArabicTreeNormalizer.normalizeWholeTree: while propagating tags down to preterminal labels (HasTag.setTag), a preterminal whose value (part-of-speech tag) is null or empty is reported as '<class>: missing tag for <pennString>'. Normalization continues, but the label will have no tag set, which can break downstream code expecting HasTag tags to be populated.","triggerScenarios":"Calling ArabicTreeNormalizer.normalizeWholeTree on an Arabic parse tree that contains preterminals with null/empty node values — typically from faulty parsers, hand-corrected treebank edits, or ATB trees imported with missing POS tags.","commonSituations":"Preprocessing Arabic Treebank data for training/parsing (ArabicStanfordSentiment, ATB pipelines); converting trees from other formats (e.g. MADA output) where some leaves lack tags.","solutions":["Inspect the logged pennString to find the offending node and fix the tree source","Assign a placeholder tag (e.g. 'NN' or 'DUMMYTAG') to preterminals before normalization","Validate input trees programmatically (check every preterminal has a non-empty value) before calling normalizeWholeTree","Regenerate the tree from the parser if the empty tag resulted from a decode/export bug"],"exampleFix":"// before\nTree t = tree; // some preterminal with value == null\nnormalizer.normalizeWholeTree(t, tf);\n// after\nfor (Tree pre : t.preTerminals()) {\n  if (pre.value() == null || pre.value().isEmpty()) pre.setValue(\"NN\"); // or DUMMYTAG placeholder\n}\nnormalizer.normalizeWholeTree(t, tf);","handlingStrategy":"validation","validationCode":"// reject/repair trees with empty preterminal tags before normalization\nboolean allTagged = true;\nfor (Tree pre : tree.preTerminals()) {\n  if (pre.value() == null || pre.value().isEmpty()) { allTagged = false; break; }\n}\nif (!allTagged) log.warn(\"Tree has untagged preterminals; fix before normalizeWholeTree\");","typeGuard":null,"tryCatchPattern":"try {\n  Tree out = normalizer.normalizeWholeTree(tree, tf);\n} catch (Exception e) {\n  log.warn(\"Normalization failed\", e);\n}","preventionTips":["Validate ATB/MADA-converted trees for null/empty POS values at load time","Fix tree sources rather than relying on the warning path","Keep the warning log enabled; missing tags silently degrade downstream HasTag consumers"],"tags":["java","nlp","trees","arabic","treebank"],"backgroundTag":"empty-required-field","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}