stanfordnlp/CoreNLP · error · IllegalArgumentException

TokensRegexNERAnnotator ERROR: Incorrectly specified options

Error message

TokensRegexNERAnnotator ERROR: Incorrectly specified options for mapping file

What it means

TokensRegexNERAnnotator parses each mapping-file entry as an optional list of key=value options followed by the file path. When the path component count (numOptions) is greater than 1, every option before the last must split on '=' into exactly two parts; if an option lacks '=' (or contains extra '=' splitting behavior producing != 2 parts), the annotator throws this IllegalArgumentException at construction time.

Solutions

Ensure every per-mapping option is written exactly as key=value, e.g. 'ignorecase=true' before the file path
Check for stray/empty segments between delimiters (double commas or trailing commas) in the mapping property
Confirm the mapping file path is the LAST item in the list and contains no option-like '=' handling issues
Quote the whole property value in your properties file/shell so delimiters are not mangled

Example fix

// before
Properties props = new Properties();
props.setProperty("tokensregexner.mapping", "ignorecase true,my_rules.tsv");
// after
props.setProperty("tokensregexner.mapping", "ignorecase=true,my_rules.tsv");

Defensive patterns

Strategy: validation

Validate before calling

String mapping = props.getProperty("tokensregexner.mapping");
if (mapping != null) {
  for (String entry : mapping.split(",")) {
    String trimmed = entry.trim();
    if (trimmed.contains("=") && trimmed.split("=").length != 2)
      throw new IllegalArgumentException("Malformed option: " + trimmed);
    if (!trimmed.equals("") && !trimmed.contains("=") && !new java.io.File(trimmed).exists())
      // could still be an option with missing '='
      if (!trimmed.matches(".*\\.(gz|txt|tsv|ser)$"))
        throw new IllegalArgumentException("Option missing '=': " + trimmed);
  }
}

Try / catch

try {
  pipeline.annotate(doc);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("Incorrectly specified options")) {
    log.error("Fix tokensregexner.mapping: every per-file option must be key=value");
  }
  throw e;
}

Prevention

Always write mapping options as key=value pairs immediately before the file path
Keep the file path as the last element of the mapping list
Validate the mapping property string in unit tests before building the pipeline
Avoid trailing/double delimiters in the property value

When it happens

Trigger: Setting the property tokensregexner.mapping.<annotatorName> (or 'mapping') with a comma/semicolon separated list where a per-file option is malformed, e.g. 'ignoreCase/mapping.ser.gz' (missing '='), 'ignoreCase=true,extra mapping.ser.gz' (empty option string), or 'a=b=c,file' where the delimiter split yields more or fewer than 2 parts.

Common situations: Typos like 'ignorecase tru' vs 'ignorecase=true', copying options from docs with wrong separators, trailing commas producing empty option segments, or mixing up the order so a path lands in the options slot.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/39f251bc3fe90e6d. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/pipeline/TokensRegexNERAnnotator.java:896

    } else {
      //Semicolons separate the files
      return SEMICOLON_DELIMITERS_PATTERN.split(mappingFiles);
    }
  }
  private static String[] processPerFileOptions(String annotatorName, String[] mappings, List<Boolean> ignoreCaseList, List<Pattern> validPosPatternList, List<String[]> headerList, boolean ignoreCase, Pattern validPosPattern, String[] headerFields, String[] annotationFieldnames, List<Class> annotationFields) {
    int numMappingFiles = mappings.length;
    for (int index = 0; index < numMappingFiles; index++) {
      boolean ignoreCaseSet = false;
      boolean validPosPatternSet = false;
      boolean headerSet = false;
      String[] allOptions = COMMA_DELIMITERS_PATTERN.split(mappings[index].trim());
      int numOptions = allOptions.length;
      String filePath = allOptions[allOptions.length - 1];
      if (numOptions > 1) { // there are some per file options here
        for (int i = 0; i < numOptions-1; i++) {
          String[] optionAndValue = EQUALS_DELIMITERS_PATTERN.split(allOptions[i].trim());
          if (optionAndValue.length != 2) {
            throw new IllegalArgumentException("TokensRegexNERAnnotator " + annotatorName
                    + " ERROR: Incorrectly specified options for mapping file " + mappings[index].trim());
          } else {
            switch (optionAndValue[0].trim().toLowerCase()) {
              case "ignorecase":
                ignoreCaseList.add(Boolean.parseBoolean(optionAndValue[1].trim()));
                ignoreCaseSet = true;
                break;
              case "validpospattern":
                String validPosRegex = optionAndValue[1].trim();
                if ( ! StringUtils.isNullOrEmpty(validPosRegex)) {
                  validPosPatternList.add(Pattern.compile(validPosRegex));
                } else {
                  validPosPatternList.add(validPosPattern);
                }
                validPosPatternSet = true;
                break;
              case "header":
                String header = optionAndValue[1].trim();

View on GitHub (pinned to 1b7edd19c4)