{"record":{"id":"6f0cf73c1247f8a6","repo":"stanfordnlp/CoreNLP","slug":"cannot-normalize-ner-tag-ner","errorCode":null,"errorMessage":"Cannot normalize ner tag ${ner}","messagePattern":"Cannot normalize ner tag (.+?)","errorType":"validation","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ie/machinereading/domains/roth/RothCONLL04Reader.java","lineNumber":81,"sourceCode":"\n  private String getNormalizedNERTag(String ner) {\n    if (ner.equalsIgnoreCase(\"O\")) {\n      return \"O\";\n    } else if (ner.equalsIgnoreCase(\"Peop\")) {\n      return \"PERSON\";\n    } else if (ner.equalsIgnoreCase(\"Loc\")) {\n      return \"LOCATION\";\n    } else if(ner.equalsIgnoreCase(\"Org\")) {\n      return \"ORGANIZATION\";\n    } else if(ner.equalsIgnoreCase(\"Other\")) {\n      return \"OTHER\";\n    } else {\n      if ( ! warnedNER) {\n        warnedNER = true;\n        logger.warning(\"This file contains NER tags not in the original Roth/Yih dataset, e.g.: \" + ner);\n      }\n    }\n    throw new RuntimeException(\"Cannot normalize ner tag \" + ner);\n  }\n\n  private Annotation readSentence(String docId, Iterator<String> lineIterator) {\n    Annotation sentence = new Annotation(\"\");\n    sentence.set(CoreAnnotations.DocIDAnnotation.class, docId);\n    sentence.set(MachineReadingAnnotations.EntityMentionsAnnotation.class, new ArrayList<>());\n    // we'll need to set things like the tokens and textContent after we've\n    // fully read the sentence\n\n    // contains the full text that we've read so far\n    StringBuilder textContent = new StringBuilder();\n    int tokenCount = 0; // how many tokens we've seen so far\n    List<CoreLabel> tokens = new ArrayList<>();\n\n    // when we've seen two blank lines in a row, this sentence is over (one\n    // blank line separates the sentence and the relations\n    int numBlankLinesSeen = 0;\n    String sentenceID = null;","sourceCodeStart":63,"sourceCodeEnd":99,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/machinereading/domains/roth/RothCONLL04Reader.java#L63-L99","documentation":"RothCONLL04Reader.getNormalizedNERTag maps CoNLL04 NER tags (Peop, Loc, Org, Other) to the library's entity types. A tag outside this set causes the method to log a one-time warning and throw this RuntimeException.","triggerScenarios":"Reading a CoNLL04-format file whose NER column contains a value not in {Peop, Loc, Org, Other} — e.g. 'Misc', 'MISC', 'PER', or empty fields from a differently flavored CoNLL file.","commonSituations":"Mixing CoNLL03 (PER/LOC/ORG/MISC) files into a CoNLL04 pipeline, files annotated with a different tag set, whitespace or BOM corrupting the tag token.","solutions":["Normalize your file's NER tags to the Roth/Yih set (Peop, Loc, Org, Other) before reading.","Extend getNormalizedNERTag to map your tag set (e.g. PER->Peop, MISC->Other).","Confirm you are reading a CoNLL04 file with the RothCONLL04Reader, not a CoNLL03 file.","Trim/strip the tag field if extra whitespace or BOM characters are present."],"exampleFix":"// before\nswitch (ner) { case \"Peop\": ... }\n// after\nif (ner.equals(\"PER\")) ner = \"Peop\";\nif (ner.equals(\"MISC\")) ner = \"Other\";","handlingStrategy":"validation","validationCode":"Set<String> valid = Set.of(\"Peop\", \"Loc\", \"Org\", \"Other\");\nif (!valid.contains(nerTag.trim())) throw new IllegalStateException(\"unknown CoNLL04 NER tag: \" + nerTag);","typeGuard":null,"tryCatchPattern":"try { dataset = RothCONLL04Reader.read(file); } catch (RuntimeException e) { logger.severe(\"bad NER tag: \" + e.getMessage()); }","preventionTips":["Convert CoNLL03-style tags (PER/ORG/LOC/MISC) to CoNLL04 tags (Peop/Org/Loc/Other) before reading.","Trim tag fields to remove whitespace/BOM.","Use the reader matching your dataset's exact tag set."],"tags":["java","stanford-nlp","conll04","ner"],"backgroundTag":"invalid-enum-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}