stanfordnlp/CoreNLP · error · IllegalArgumentException

Unable to process annotators

Error message

Unable to process annotators ${annotators}

What it means

In unifyTokenizeProperty (used by normalizeAnnotators), CoreNLP folds legacy 'whitespace'/'ssplit'-style options into the 'tokenize' annotator options. When it strips an unwanted comma-separated option and cannot find a remaining comma to splice around, it cannot rewrite the annotator list and throws IllegalArgumentException('Unable to process annotators ...').

Solutions

1. Rewrite the annotators list to use modern tokenize options instead of the separate option annotator: annotators=tokenize,ssplit with tokenize.whitespace=true.
2. Remove the deprecated option annotator name from the annotators property entirely.
3. Ensure the annotators string is well-formed (no stray/duplicate commas) before pipeline construction.

Example fix

// before
props.setProperty("annotators", "tokenize,whitespace,ssplit");
props.setProperty("tokenize.whitespace", "true");
// after
props.setProperty("annotators", "tokenize,ssplit");
props.setProperty("tokenize.whitespace", "true");

String annotators = props.getProperty("annotators", "");
if (annotators.contains("whitespace") && props.containsKey("tokenize.whitespace")) {
  props.setProperty("annotators", annotators.replace(",whitespace", "").replace("whitespace,", "").replace("whitespace", ""));
}

try {
  pipeline = new StanfordCoreNLP(props);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Unable to process annotators")) {
    props.setProperty("annotators", "tokenize,ssplit"); // drop legacy option annotators
    pipeline = new StanfordCoreNLP(props);
  } else throw e;
}

When it happens

Trigger: Setting properties like 'tokenize.whitespace=true' alongside an annotators list where the corresponding option annotator appears as the only comma-separated element (no surrounding commas), so the rewrite logic runs out of commas and fails.

Common situations: Migrating old pipelines (annotators=tokenize,whitespace,ssplit patterns or a lone 'whitespace' entry) to modern CoreNLP; programmatic construction of the annotators string with trailing/malformed commas.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• unable to find sentences in
• unable to find sentences in
• annotator " " requires annotation " ". The usual…
• Invalid metric type for
• Invalid ordering constraint

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/df9a6332318e495d. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/pipeline/StanfordCoreNLP.java:356

   * In such a case, we remove the cleanxml from the annotators and set
   * the tokenize.cleanxml option instead
   */
  static void unifyTokenizeProperty(Properties properties, String property, String option) {
    String annotators = properties.getProperty("annotators", "");
    int tokenize = annotators.indexOf(STANFORD_TOKENIZE);
    int unwanted = annotators.indexOf(property);

    if (unwanted >= 0 && tokenize >= 0) {
      if (option != null) {
        properties.setProperty(option, "true");
      }
      int comma = annotators.indexOf(",", unwanted);
      if (comma >= 0) {
        annotators = annotators.substring(0, unwanted) + annotators.substring(comma+1);
      } else {
        comma = annotators.lastIndexOf(",");
        if (comma < 0) {
          throw new IllegalArgumentException("Unable to process annotators " + annotators);
        }
        annotators = annotators.substring(0, comma);
      }
      if (option != null) {
        logger.debug(property + " can now be triggered as an option to tokenize rather than a separate annotator via " + option + "=true");
      } else {
        logger.debug(property + " is now included as part of the tokenize annotator by default");
      }
      logger.debug("Updating annotators from " + properties.getProperty("annotators") + " to " + annotators);
      properties.setProperty("annotators", annotators);
    }
  }

  //
  // @Override-able methods to change pipeline behavior
  //

  /**

View on GitHub (pinned to 1b7edd19c4)