stanfordnlp/CoreNLP · error · IllegalArgumentException

SpanishLexer: the invertible option requires a…

Error message

SpanishLexer: the invertible option requires a CoreLabelTokenFactory

What it means

The invertible option of SpanishLexer requires the token factory to be a CoreLabelTokenFactory, because invertibility (recording original character offsets / before-text) needs CoreLabel tokens. If invertible is requested with any other LexedTokenFactory (e.g. WhitespaceTokenFactory or a WordTokenFactory), the constructor throws this IllegalArgumentException.

Solutions

Pass a CoreLabelTokenFactory as the token factory when invertible=true.
Disable the invertible option if CoreLabel output (and offsets) are not needed.
When using CoreNLP properties, ensure tokenize.invertible is false or that the language pipeline uses CoreLabelTokenFactory.

Example fix

// before
new SpanishLexer("invertible=true", false, new WhitespaceTokenFactory());
// after
new SpanishLexer("invertible=true", false, new CoreLabelTokenFactory());

Defensive patterns

Strategy: validation

Validate before calling

if (invertible && !(tokenFactory instanceof CoreLabelTokenFactory)) {
  throw new IllegalArgumentException("invertible=true requires CoreLabelTokenFactory");
}

Type guard

boolean canBeInvertible = (tf instanceof CoreLabelTokenFactory);

Try / catch

try {
  lexer = new SpanishLexer("invertible=true", false, tf);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("invertible")) {
    lexer = new SpanishLexer("invertible=false", false, tf); // fallback without offsets
  } else throw e;
}

Prevention

Default to CoreLabelTokenFactory whenever you need character offsets or invertibility.
Keep the invertible flag and the token factory type in the same configuration object so they cannot drift apart.
Document that invertible implies CoreLabel output in your project's tokenizer wrapper.

When it happens

Trigger: new SpanishLexer("invertible=true", false, someNonCoreLabelTokenFactory) - i.e. requesting invertible=true while passing a token factory that is not an instance of CoreLabelTokenFactory.

Common situations: Configuring CoreNLP with tokenize.invertible=true but a tokenize.language/token factory combination that does not use CoreLabelTokenFactory; custom code reusing a WordTokenFactory built for plain tokenization.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/b4ed25c56a2db77d. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/international/spanish/process/SpanishLexer.java:14194

              case "firstKeep":
                untokenizable = UntokenizableOptions.FIRST_KEEP;
                break;
              case "allKeep":
                untokenizable = UntokenizableOptions.ALL_KEEP;
                break;
              default:
                throw new IllegalArgumentException("SpanishLexer: Invalid option value in constructor: " + key + ": " + value);
            }
          } else if ("strictTreebank3".equals(key)) {
            strictTreebank3 = val;
          } else {
            throw new IllegalArgumentException(String.format("%s: Invalid options key in constructor: %s%n", this.getClass().getName(), key));
          }
        }
        // this.seenUntokenizableCharacter = false; // unnecessary, it's default initialized
        if (invertible) {
          if ( ! (tf instanceof CoreLabelTokenFactory)) {
            throw new IllegalArgumentException("SpanishLexer: the invertible option requires a CoreLabelTokenFactory");
          }
          prevWord = (CoreLabel) tf.makeToken("", 0, 0);
          prevWordAfter = new StringBuilder();
        }
      }


      /** Turn on to find out how things were tokenized. */
      private static final boolean DEBUG = false;

      /** A logger for this class */
      private static final Redwood.RedwoodChannels logger = Redwood.channels(SpanishLexer.class);

      private LexedTokenFactory<?> tokenFactory;
      private CoreLabel prevWord;
      private StringBuilder prevWordAfter;
      private boolean seenUntokenizableCharacter;
      private enum UntokenizableOptions { NONE_DELETE, FIRST_DELETE, ALL_DELETE, NONE_KEEP, FIRST_KEEP, ALL_KEEP }

View on GitHub (pinned to 1b7edd19c4)