stanfordnlp/CoreNLP · error · IllegalArgumentException

FrenchLexer: the invertible option requires a…

Error message

FrenchLexer: the invertible option requires a CoreLabelTokenFactory

What it means

When the 'invertible' option is enabled, FrenchLexer must record before/after text around each token, which requires a CoreLabelTokenFactory. If the supplied TokenFactory is not a CoreLabelTokenFactory, the constructor throws this IllegalArgumentException immediately.

Solutions

1. Pass a CoreLabelTokenFactory when using invertible=true
2. Remove the invertible option if token before/after strings are not needed
3. Ensure the factory instance (not just any TokenFactory) is CoreLabelTokenFactory

Example fix

// before
new FrenchLexer(options, reader, new WordTokenFactory(), true);
// after
new FrenchLexer(options, reader, new CoreLabelTokenFactory(), true);

if (options.contains("invertible=true") && !(tf instanceof CoreLabelTokenFactory)) throw new IllegalStateException("invertible requires CoreLabelTokenFactory");

boolean supportsInvertible(LexedTokenFactory tf) { return tf instanceof CoreLabelTokenFactory; }

try { lexer = new FrenchLexer(options, reader, tf); } catch (IllegalArgumentException e) { log.error("Factory/option conflict: " + e.getMessage()); lexer = new FrenchLexer(options, reader, new CoreLabelTokenFactory()); }

When it happens

Trigger: Constructing FrenchLexer with options containing 'invertible=true' while passing a LexedTokenFactory that is not a CoreLabelTokenFactory (e.g. WordTokenFactory).

Common situations: Using FrenchLexer inside a pipeline configured with a custom token factory; switching token factories for efficiency while keeping invertible enabled for token-before/after information.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Not a valid ellipses style:
• Not a valid dashes style:
• FrenchLexer: Invalid option value in constructor: :
• Not a valid ellipses style
• Not a valid dashes style

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/8540d31fcaeccf56. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/international/french/process/FrenchLexer.java:12452

            case "firstKeep":
              untokenizable = UntokenizableOptions.FIRST_KEEP;
              break;
            case "allKeep":
              untokenizable = UntokenizableOptions.ALL_KEEP;
              break;
            default:
              throw new IllegalArgumentException("FrenchLexer: Invalid option value in constructor: " + key + ": " + value);
          }
        } else if ("strictTreebank3".equals(key)) {
          strictTreebank3 = val;
        } else {
          System.err.printf("%s: Invalid options key in constructor: %s%n", this.getClass().getName(), key);
        }
      }
      // this.seenUntokenizableCharacter = false; // unnecessary, it's default initialized
      if (invertible) {
        if ( ! (tf instanceof CoreLabelTokenFactory)) {
          throw new IllegalArgumentException("FrenchLexer: the invertible option requires a CoreLabelTokenFactory");
        }
        prevWord = (CoreLabel) tf.makeToken("", 0, 0);
        prevWordAfter = new StringBuilder();
      }
    }


    /** Turn on to find out how things were tokenized. */
    private static final boolean DEBUG = false;

    /** A logger for this class */
    private static final Redwood.RedwoodChannels logger = Redwood.channels(FrenchLexer.class);


    private LexedTokenFactory<?> tokenFactory;
    private CoreLabel prevWord;
    private StringBuilder prevWordAfter;
    private boolean seenUntokenizableCharacter;

View on GitHub (pinned to 1b7edd19c4)