: Replacing duplicate entry (higher priority): old= , new=

Error message

${annotatorName}: Replacing duplicate entry (higher priority): old=${oldEntry}, new=${entry}

What it means

When a TokensRegexNER rule file contains two entries with the same regex key, the annotator keeps one: if the new entry has a strictly higher priority, it replaces the old one and this warning is logged showing both entries. This prevents silent duplication but signals your rule file has duplicate keys.

Solutions

Search the mapping file for duplicate keys and delete the superseded entry
If both rules are intended, differentiate the regex keys or adjust priorities deliberately
Keep rules files deduplicated programmatically (e.g. sort/unique on the first column) as a build step

Example fix

// before (mapping file)
Barack	PERSON	0.0
Barack	PERSON	1.0
// after
Barack	PERSON	1.0

Defensive patterns

Strategy: validation

Validate before calling

Set<String> seen = new HashSet<>();
for (String line : Files.readAllLines(rulesPath)) {
  String key = line.split("\t")[0];
  if (!seen.add(key))
    throw new IllegalArgumentException("Duplicate rule key: " + key);
}

Prevention

Deduplicate rule keys before shipping rules files
Merge rule sources deliberately, not by concatenation
Keep a canonical rules file under version control
Watch startup warnings for duplicate-entry messages in CI

When it happens

Trigger: The same tokens regex (key) appears twice in a mapping file with different priorities; the later, higher-priority entry overwrites the earlier one at load time.

Common situations: Appending new rules to an existing file without removing old duplicates; merging rule files from different teams; copy-paste duplication in hand-maintained gazetteers.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/789abc8b23e182fd. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/pipeline/TokensRegexNERAnnotator.java:825

      for (int i = 0; i < types.length; i++) {
        String type = types[i];
        // TODO: Have option to allow commas in types
        int commaPos = type.indexOf(',');
        if (commaPos > 0) {
          // Strip the "," and just take first type
          String newType = type.substring(0, commaPos).trim();
          logger.warn(annotatorName + ": Entry has multiple types for " +
                  annotationFieldnames[i] + ": " + line + ".  Taking type to be " + newType);
          types[i] = newType;
        }
      }

      Entry entry = new Entry(tokensRegex, regexes, types, overwritableTypes, priority, weight, annotateGroup);

      if (seenRegexes.containsKey(Arrays.asList(key))) {
        Entry oldEntry = seenRegexes.get(key);
        if (priority > oldEntry.priority) {
          logger.warn(annotatorName +
                  ": Replacing duplicate entry (higher priority): old=" + oldEntry + ", new=" + entry);
        } else {
          String oldTypeDesc = oldEntry.getTypeDescription();
          String newTypeDesc = entry.getTypeDescription();
          if (!oldTypeDesc.equals(newTypeDesc)) {
            if (verbose) {
              logger.warn(annotatorName + ": Ignoring duplicate entry: " +
                      split[0] + ", old type = " + oldTypeDesc + ", new type = " + newTypeDesc);
            }
          // } else {
          //   if (verbose) {
          //     logger.warn(annotatorName + ": Duplicate entry [ignored]: " +
          //             split[0] + ", old type = " + oldEntry.type + ", new type = " + type);
          //   }
          }
          continue;
        }
      }

View on GitHub (pinned to 1b7edd19c4)