{"record":{"id":"2804ef07b9055542","repo":"stanfordnlp/CoreNLP","slug":"annotatorname-ignoring-duplicate-entry-spli","errorCode":null,"errorMessage":"${annotatorName}: Ignoring duplicate entry: ${split[0]}, old type = ${oldTypeDesc}, new type = ${newTypeDesc}","messagePattern":"(.+?): Ignoring duplicate entry: (.+?), old type = (.+?), new type = (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"info","filePath":"src/edu/stanford/nlp/pipeline/TokensRegexNERAnnotator.java","lineNumber":832,"sourceCode":"          logger.warn(annotatorName + \": Entry has multiple types for \" +\n                  annotationFieldnames[i] + \": \" + line + \".  Taking type to be \" + newType);\n          types[i] = newType;\n        }\n      }\n\n      Entry entry = new Entry(tokensRegex, regexes, types, overwritableTypes, priority, weight, annotateGroup);\n\n      if (seenRegexes.containsKey(Arrays.asList(key))) {\n        Entry oldEntry = seenRegexes.get(key);\n        if (priority > oldEntry.priority) {\n          logger.warn(annotatorName +\n                  \": Replacing duplicate entry (higher priority): old=\" + oldEntry + \", new=\" + entry);\n        } else {\n          String oldTypeDesc = oldEntry.getTypeDescription();\n          String newTypeDesc = entry.getTypeDescription();\n          if (!oldTypeDesc.equals(newTypeDesc)) {\n            if (verbose) {\n              logger.warn(annotatorName + \": Ignoring duplicate entry: \" +\n                      split[0] + \", old type = \" + oldTypeDesc + \", new type = \" + newTypeDesc);\n            }\n          // } else {\n          //   if (verbose) {\n          //     logger.warn(annotatorName + \": Duplicate entry [ignored]: \" +\n          //             split[0] + \", old type = \" + oldEntry.type + \", new type = \" + type);\n          //   }\n          }\n          continue;\n        }\n      }\n\n      // Print some warning if label belongs to noDefaultOverwriteLabels but there is no overwritable types\n      if (entry.overwritableTypes.isEmpty() && hasNoOverwritableType(noDefaultOverwriteLabels, entry.types)) {\n        logger.warn(annotatorName + \": Entry doesn't have overwriteable types \" +\n                entry + \", but entry type is in noDefaultOverwriteLabels\");\n      }\n","sourceCodeStart":814,"sourceCodeEnd":850,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/pipeline/TokensRegexNERAnnotator.java#L814-L850","documentation":"When a duplicate TokensRegexNER entry is seen and the new entry's priority is NOT higher, the old entry is kept. If the old and new entries annotate different types, this message (logged only when verbose is enabled) reports that the duplicate was ignored, showing both type descriptions. Functionally it is a suppressed duplicate with a differing type.","triggerScenarios":"A mapping file contains the same regex key twice with equal or lower priority but a different NER type; verbose=true for the TokensRegexNERAnnotator so the message is actually emitted.","commonSituations":"Merged gazetteer files where the same name is typed differently (PERSON vs ORGANIZATION); team members independently adding conflicting entries for the same entity.","solutions":["Remove the duplicate line from the mapping file, keeping the entry with the desired type/priority","Raise the duplicate entry's priority above the existing one if the new type is correct","Enable verbose only during debugging; suppress this noise in production","Add deduplication checks to your rules-file build process"],"exampleFix":"// before (mapping file)\nAmazon\tORGANIZATION\t0.0\nAmazon\tPERSON\t0.0\n// after\nAmazon\tORGANIZATION\t0.0","handlingStrategy":"validation","validationCode":"Map<String,String> keyToType = new HashMap<>();\nfor (String line : Files.readAllLines(rulesPath)) {\n  String[] parts = line.split(\"\\t\");\n  String prev = keyToType.putIfAbsent(parts[0], parts.length > 1 ? parts[1] : \"\");\n  if (prev != null && !prev.equals(parts.length > 1 ? parts[1] : \"\"))\n    throw new IllegalArgumentException(\"Conflicting types for key: \" + parts[0]);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Resolve conflicting types for the same entity in source files","Use priorities intentionally when duplicates are legitimate","Enable verbose only when debugging rule loading","Deduplicate and reconcile gazetteers as a build step"],"tags":["nlp","ner","duplicate-keys","configuration"],"backgroundTag":"duplicate-entry","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}