: Ignoring duplicate entry: , old type = , new type =

Error message

${annotatorName}: Ignoring duplicate entry: ${split[0]}, old type = ${oldTypeDesc}, new type = ${newTypeDesc}

What it means

When a duplicate TokensRegexNER entry is seen and the new entry's priority is NOT higher, the old entry is kept. If the old and new entries annotate different types, this message (logged only when verbose is enabled) reports that the duplicate was ignored, showing both type descriptions. Functionally it is a suppressed duplicate with a differing type.

Solutions

Remove the duplicate line from the mapping file, keeping the entry with the desired type/priority
Raise the duplicate entry's priority above the existing one if the new type is correct
Enable verbose only during debugging; suppress this noise in production
Add deduplication checks to your rules-file build process

Example fix

// before (mapping file)
Amazon	ORGANIZATION	0.0
Amazon	PERSON	0.0
// after
Amazon	ORGANIZATION	0.0

Defensive patterns

Strategy: validation

Validate before calling

Map<String,String> keyToType = new HashMap<>();
for (String line : Files.readAllLines(rulesPath)) {
  String[] parts = line.split("\t");
  String prev = keyToType.putIfAbsent(parts[0], parts.length > 1 ? parts[1] : "");
  if (prev != null && !prev.equals(parts.length > 1 ? parts[1] : ""))
    throw new IllegalArgumentException("Conflicting types for key: " + parts[0]);
}

Prevention

Resolve conflicting types for the same entity in source files
Use priorities intentionally when duplicates are legitimate
Enable verbose only when debugging rule loading
Deduplicate and reconcile gazetteers as a build step

When it happens

Trigger: A mapping file contains the same regex key twice with equal or lower priority but a different NER type; verbose=true for the TokensRegexNERAnnotator so the message is actually emitted.

Common situations: Merged gazetteer files where the same name is typed differently (PERSON vs ORGANIZATION); team members independently adding conflicting entries for the same entity.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/2804ef07b9055542. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/pipeline/TokensRegexNERAnnotator.java:832

          logger.warn(annotatorName + ": Entry has multiple types for " +
                  annotationFieldnames[i] + ": " + line + ".  Taking type to be " + newType);
          types[i] = newType;
        }
      }

      Entry entry = new Entry(tokensRegex, regexes, types, overwritableTypes, priority, weight, annotateGroup);

      if (seenRegexes.containsKey(Arrays.asList(key))) {
        Entry oldEntry = seenRegexes.get(key);
        if (priority > oldEntry.priority) {
          logger.warn(annotatorName +
                  ": Replacing duplicate entry (higher priority): old=" + oldEntry + ", new=" + entry);
        } else {
          String oldTypeDesc = oldEntry.getTypeDescription();
          String newTypeDesc = entry.getTypeDescription();
          if (!oldTypeDesc.equals(newTypeDesc)) {
            if (verbose) {
              logger.warn(annotatorName + ": Ignoring duplicate entry: " +
                      split[0] + ", old type = " + oldTypeDesc + ", new type = " + newTypeDesc);
            }
          // } else {
          //   if (verbose) {
          //     logger.warn(annotatorName + ": Duplicate entry [ignored]: " +
          //             split[0] + ", old type = " + oldEntry.type + ", new type = " + type);
          //   }
          }
          continue;
        }
      }

      // Print some warning if label belongs to noDefaultOverwriteLabels but there is no overwritable types
      if (entry.overwritableTypes.isEmpty() && hasNoOverwritableType(noDefaultOverwriteLabels, entry.types)) {
        logger.warn(annotatorName + ": Entry doesn't have overwriteable types " +
                entry + ", but entry type is in noDefaultOverwriteLabels");
      }

View on GitHub (pinned to 1b7edd19c4)