What causes "%s: missing tag for %s" in stanfordnlp/CoreNLP?

Inspect the logged pennString to find the offending node and fix the tree source

: missing tag for

Error message

%s: missing tag for %s

What it means

A warning (not exception) from ArabicTreeNormalizer.normalizeWholeTree: while propagating tags down to preterminal labels (HasTag.setTag), a preterminal whose value (part-of-speech tag) is null or empty is reported as '<class>: missing tag for <pennString>'. Normalization continues, but the label will have no tag set, which can break downstream code expecting HasTag tags to be populated.

Solutions

Inspect the logged pennString to find the offending node and fix the tree source
Assign a placeholder tag (e.g. 'NN' or 'DUMMYTAG') to preterminals before normalization
Validate input trees programmatically (check every preterminal has a non-empty value) before calling normalizeWholeTree
Regenerate the tree from the parser if the empty tag resulted from a decode/export bug

Example fix

// before
Tree t = tree; // some preterminal with value == null
normalizer.normalizeWholeTree(t, tf);
// after
for (Tree pre : t.preTerminals()) {
  if (pre.value() == null || pre.value().isEmpty()) pre.setValue("NN"); // or DUMMYTAG placeholder
}
normalizer.normalizeWholeTree(t, tf);

Defensive patterns

Strategy: validation

Validate before calling

// reject/repair trees with empty preterminal tags before normalization
boolean allTagged = true;
for (Tree pre : tree.preTerminals()) {
  if (pre.value() == null || pre.value().isEmpty()) { allTagged = false; break; }
}
if (!allTagged) log.warn("Tree has untagged preterminals; fix before normalizeWholeTree");

Try / catch

try {
  Tree out = normalizer.normalizeWholeTree(tree, tf);
} catch (Exception e) {
  log.warn("Normalization failed", e);
}

Prevention

Validate ATB/MADA-converted trees for null/empty POS values at load time
Fix tree sources rather than relying on the warning path
Keep the warning log enabled; missing tags silently degrade downstream HasTag consumers

When it happens

Trigger: Calling ArabicTreeNormalizer.normalizeWholeTree on an Arabic parse tree that contains preterminals with null/empty node values — typically from faulty parsers, hand-corrected treebank edits, or ATB trees imported with missing POS tags.

Common situations: Preprocessing Arabic Treebank data for training/parsing (ArabicStanfordSentiment, ATB pipelines); converting trees from other formats (e.g. MADA output) where some leaves lack tags.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/4d14ba4d562fb41f. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/trees/international/arabic/ArabicTreeNormalizer.java:162

            } else {
              // TODO(spenceg): Does this help?
              String newLemma = lexMapper.map(null, lemma);
              if (newLemma == null || newLemma.trim().isEmpty()) {
                newLemma = lemma;
              }
              String newMorphAnalysis = newLemma + MorphoFeatureSpecification.LEMMA_MARK + morphAnalysis;
              cl.setOriginalText(newMorphAnalysis.intern());
            }

          } else {
            log.error(String.format("%s: Cannot store morph analysis in non-CoreLabel: %s",this.getClass().getName(),t.label().getClass().getName()));
          }
        }

      } else if (t.isPreTerminal()) {

        if (t.value() == null || t.value().isEmpty()) {
          log.warn(String.format("%s: missing tag for %s",this.getClass().getName(),t.pennString()));
        } else if(t.label() instanceof HasTag) {
          ((HasTag) t.label()).setTag(t.value());
        }

      } else { //Phrasal nodes

        // there are some nodes "/" missing preterminals.  We'll splice in a tag for these.
        int nk = t.numChildren();
        List<Tree> newKids = new ArrayList<>(nk);
        for (int j = 0; j < nk; j++) {
          Tree child = t.getChild(j);
          if (child.isLeaf()) {
            log.warn(String.format("%s: Splicing in DUMMYTAG for %s",this.getClass().getName(),t.toString()));
            newKids.add(tf.newTreeNode("DUMMYTAG", Collections.singletonList(child)));

          } else {
            newKids.add(child);
          }

View on GitHub (pinned to 1b7edd19c4)