stanfordnlp/CoreNLP · error · RuntimeException

Cannot normalize ner tag

Error message

Cannot normalize ner tag ${ner}

What it means

RothCONLL04Reader.getNormalizedNERTag maps CoNLL04 NER tags (Peop, Loc, Org, Other) to the library's entity types. A tag outside this set causes the method to log a one-time warning and throw this RuntimeException.

Solutions

1. Normalize your file's NER tags to the Roth/Yih set (Peop, Loc, Org, Other) before reading.
2. Extend getNormalizedNERTag to map your tag set (e.g. PER->Peop, MISC->Other).
3. Confirm you are reading a CoNLL04 file with the RothCONLL04Reader, not a CoNLL03 file.
4. Trim/strip the tag field if extra whitespace or BOM characters are present.

Example fix

// before
switch (ner) { case "Peop": ... }
// after
if (ner.equals("PER")) ner = "Peop";
if (ner.equals("MISC")) ner = "Other";

Set<String> valid = Set.of("Peop", "Loc", "Org", "Other");
if (!valid.contains(nerTag.trim())) throw new IllegalStateException("unknown CoNLL04 NER tag: " + nerTag);

try { dataset = RothCONLL04Reader.read(file); } catch (RuntimeException e) { logger.severe("bad NER tag: " + e.getMessage()); }

When it happens

Trigger: Reading a CoNLL04-format file whose NER column contains a value not in {Peop, Loc, Org, Other} — e.g. 'Misc', 'MISC', 'PER', or empty fields from a differently flavored CoNLL file.

Common situations: Mixing CoNLL03 (PER/LOC/ORG/MISC) files into a CoNLL04 pipeline, files annotated with a different tag set, whitespace or BOM corrupting the tag token.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Error: a relation was marked between two words where one of…
• Array lengths don't match
• Attempt to use ExternalFiniteDifference without passing…
• Attempting to remove features based on weight from a…
• Bad data format:

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/6f0cf73c1247f8a6. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/ie/machinereading/domains/roth/RothCONLL04Reader.java:81

  private String getNormalizedNERTag(String ner) {
    if (ner.equalsIgnoreCase("O")) {
      return "O";
    } else if (ner.equalsIgnoreCase("Peop")) {
      return "PERSON";
    } else if (ner.equalsIgnoreCase("Loc")) {
      return "LOCATION";
    } else if(ner.equalsIgnoreCase("Org")) {
      return "ORGANIZATION";
    } else if(ner.equalsIgnoreCase("Other")) {
      return "OTHER";
    } else {
      if ( ! warnedNER) {
        warnedNER = true;
        logger.warning("This file contains NER tags not in the original Roth/Yih dataset, e.g.: " + ner);
      }
    }
    throw new RuntimeException("Cannot normalize ner tag " + ner);
  }

  private Annotation readSentence(String docId, Iterator<String> lineIterator) {
    Annotation sentence = new Annotation("");
    sentence.set(CoreAnnotations.DocIDAnnotation.class, docId);
    sentence.set(MachineReadingAnnotations.EntityMentionsAnnotation.class, new ArrayList<>());
    // we'll need to set things like the tokens and textContent after we've
    // fully read the sentence

    // contains the full text that we've read so far
    StringBuilder textContent = new StringBuilder();
    int tokenCount = 0; // how many tokens we've seen so far
    List<CoreLabel> tokens = new ArrayList<>();

    // when we've seen two blank lines in a row, this sentence is over (one
    // blank line separates the sentence and the relations
    int numBlankLinesSeen = 0;
    String sentenceID = null;

View on GitHub (pinned to 1b7edd19c4)