stanfordnlp/CoreNLP · error · IllegalArgumentException

TokensRegexNERAnnotator ERROR: Header does not contain…

Error message

TokensRegexNERAnnotator ERROR: Header does not contain annotation field '':

What it means

After validating 'pattern', readEntries resolves each configured annotation field name (default 'ner') against the header index map; if any annotation field is missing from the header it throws IllegalArgumentException listing the full header. Every column the annotator is told to write must exist in the mapping file header.

Solutions

Rename the mapping-file column to match the configured annotation field (default 'ner').
Or set the tokensregexner.annotationFieldnames property to match the actual header column names.
Add the missing column to the header and populate it per row.
Check exact spelling/case of both header and configured field names.

Example fix

// before: header 'pattern\ttype', default annotator expects 'ner'
// after
props.setProperty("tokensregexner.annotationFieldnames", "type");
// or rename the column: pattern\tner

Defensive patterns

Strategy: validation

Validate before calling

Set<String> header = new LinkedHashSet<>(Arrays.asList(firstLine.split("\t", -1)));
for (String field : annotationFieldnames) {
  if (!header.contains(field)) throw new IllegalStateException("Header missing annotation field '" + field + "'");
}

Try / catch

try { annotator = new TokensRegexNERAnnotator(name, props); } catch (IllegalArgumentException e) { if (e.getMessage().contains("Header does not contain annotation field")) { alignHeaderWithConfig(mappingPath, props); } else throw e; }

Prevention

Keep annotationFieldnames options and the header columns in sync; change them together
Use the default field name 'ner' unless deliberately renaming columns
Add a startup smoke test that constructs the annotator from the production mapping file

When it happens

Trigger: Setting tokensregexner.annotationFieldnames (or the default 'ner' field) to a name absent from the header row — e.g. header is 'pattern type' but the annotator expects 'ner'; or requesting extra fields like 'normalized' that the header never declares.

Common situations: Renaming a column in the mapping file without updating the annotator options; enabling additional annotation fields in properties without extending the header; default 'ner' expected but header uses 'label' or 'tag'.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/595528dba90edc17. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/pipeline/TokensRegexNERAnnotator.java:685

    int origEntriesSize = entries.size();
    int isTokensRegex = 0;
    int lineCount = 0;
    Map<String,Integer> headerIndexMap = getHeaderIndexMap(headerFields);
    int iPattern = getIndex(headerIndexMap, PATTERN_FIELD);
    if (iPattern < 0) {
      throw new IllegalArgumentException("TokensRegexNERAnnotator " + annotatorName
        + " ERROR: Header does not contain 'pattern': " + StringUtils.join(headerFields));
    }
    int iOverwrite = getIndex(headerIndexMap, OVERWRITE_FIELD);
    int iPriority = getIndex(headerIndexMap, PRIORITY_FIELD);
    int iWeight = getIndex(headerIndexMap, WEIGHT_FIELD);
    int iGroup = getIndex(headerIndexMap, GROUP_FIELD);
    int[] annotationCols = new int[annotationFieldnames.length];
    int iLastAnnotationField = -1;
    for (int i = 0; i < annotationFieldnames.length; i++) {
      annotationCols[i] = getIndex(headerIndexMap, annotationFieldnames[i]);
      if (annotationCols[i] < 0) {
        throw new IllegalArgumentException("TokensRegexNERAnnotator " + annotatorName
            + " ERROR: Header does not contain annotation field '" + annotationFieldnames[i] + "': " + StringUtils.join(headerFields));
      }
      if (annotationCols[i] > iLastAnnotationField) {
        iLastAnnotationField = annotationCols[i];
      }
    }

    // Take minimum of "pattern" and last annotation field; add one to it to map array index to minimum length
    int minLength = Math.max(iPattern, iLastAnnotationField) + 1;
    int maxLength = headerFields.length;  // Take maximum number of headerFields
    for (String line; (line = mapping.readLine()) != null; ) {
      lineCount ++;
      String[] split = line.split("\t");

      if (lineCount == 1) {
        if (split.length == headerFields.length) {
          boolean equals = true;
          for (int i = 0; i < split.length; i ++) {

View on GitHub (pinned to 1b7edd19c4)