{"record":{"id":"66031a47c8e338b3","repo":"stanfordnlp/CoreNLP","slug":"error-with-entity-construction-two-tokens-had-inc","errorCode":null,"errorMessage":"Error with entity construction, two tokens had inconsistent NER tags","messagePattern":"Error with entity construction, two tokens had inconsistent NER tags","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ie/ChineseQuantifiableEntityNormalizer.java","lineNumber":934,"sourceCode":"\n    return s;\n  }\n\n  /**\n   * Concatenate entity annotations to a String. Note that Chinese does not use space to separate\n   * tokens so we will follow this convention here.\n   *\n   * @param l\n   * @param <E>\n   * @return\n   */\n  private static <E extends CoreMap> String singleEntityToString(List<E> l) {\n    String entityType = l.get(0).get(CoreAnnotations.NamedEntityTagAnnotation.class);\n    StringBuilder sb = new StringBuilder();\n    for (E w : l) {\n      if(!w.get(CoreAnnotations.NamedEntityTagAnnotation.class).equals(entityType)) {\n        log.error(\"differing NER tags detected in entity: \" + l);\n        throw new Error(\"Error with entity construction, two tokens had inconsistent NER tags\");\n      }\n      sb.append(w.get(CoreAnnotations.TextAnnotation.class));\n    }\n    return sb.toString();\n  }\n\n  private static String prettyNumber(String s) {\n    if (s == null) {\n      return null;\n    }\n    s = ! s.contains(\".\") ? s : s.replaceAll(\"0*$\", \"\").replaceAll(\"\\\\.$\", \"\");\n    return s;\n  }\n\n\n  /**\n   * Fix up the NER sequence in case this is necessary.\n   *","sourceCodeStart":916,"sourceCodeEnd":952,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/ChineseQuantifiableEntityNormalizer.java#L916-L952","documentation":"ChineseQuantifiableEntityNormalizer concatenates tokens of one entity before normalizing it. In singleEntityToString it verifies every token in the List<CoreMap> carries the same NamedEntityTagAnnotation; if the tags differ, the 'entity' was constructed incorrectly and it throws an Error rather than producing garbage output.","triggerScenarios":"Calling normalize/normalizeString (or running the NER pipeline feeding it) on a List<CoreMap> entity where tokens have inconsistent NamedEntityTagAnnotation values — e.g. a merge of adjacent tokens with different NER labels, or hand-built CoreMaps missing the tag on some tokens.","commonSituations":"Custom post-processing that splices tokens from different NER-labeled sentences into one entity list; CoreLabel lists built manually where the NER tag was only set on the first token; bugs in upstream entity merging in a customized pipeline.","solutions":["Inspect the entity list logged by 'differing NER tags detected in entity:' and fix the code that built/merged it so all tokens share one NER tag","Ensure every CoreLabel passed in has NamedEntityTagAnnotation set (call set(NamedEntityTagAnnotation.class, ...) on all tokens)","Check for and disable custom NER post-processors/mergers that relabel individual tokens within an entity","If intentional mixed input is expected, split the list into entities before calling the normalizer"],"exampleFix":"// before: entity list with mixed tags\nList<CoreLabel> entity = mergeTokens(nerTokenA, plainTokenB); // tokenB has tag null/O\nnormalizer.normalize(entity); // throws\n// after: set a consistent tag on all tokens\nfor (CoreLabel tok : entity) tok.set(CoreAnnotations.NamedEntityTagAnnotation.class, \"NUMBER\");\nnormalizer.normalize(entity);","handlingStrategy":"validation","validationCode":"Set<String> tags = entity.stream()\n    .map(w -> w.get(CoreAnnotations.NamedEntityTagAnnotation.class))\n    .filter(Objects::nonNull)\n    .collect(Collectors.toSet());\nif (tags.size() != 1) throw new IllegalStateException(\"inconsistent NER tags: \" + tags);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always set NamedEntityTagAnnotation on every token you hand to the normalizer","Don't merge tokens across different NER-labeled segments","Log the offending entity list (as the library does) to find the builder bug"],"tags":["java","ner","corenlp","entity-construction"],"backgroundTag":"internal-invariant-violation","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}