{"record":{"id":"f4025ae53e5995a8","repo":"stanfordnlp/CoreNLP","slug":"ptblexer-invalid-options-key-in-constructor-ke","errorCode":null,"errorMessage":"PTBLexer: Invalid options key in constructor: ${key}","messagePattern":"PTBLexer: Invalid options key in constructor: (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/process/PTBLexer.java","lineNumber":61176,"sourceCode":"                break;\n              case \"firstKeep\":\n                untokenizable = UntokenizableOptions.FIRST_KEEP;\n                break;\n              case \"allKeep\":\n                untokenizable = UntokenizableOptions.ALL_KEEP;\n                break;\n              default:\n                throw new IllegalArgumentException(\"PTBLexer: Invalid option value in constructor: \" + key + \": \" + value);\n            }\n          } else if (\"strictTreebank3\".equals(key)) {\n            strictFraction = val;\n            strictAcronym = val;\n          } else if (\"strictFraction\".equals(key)) {\n            strictFraction = val;\n          } else if (\"strictAcronym\".equals(key)) {\n            strictAcronym = val;\n          } else {\n            throw new IllegalArgumentException(\"PTBLexer: Invalid options key in constructor: \" + key);\n          }\n        }\n        if (invertible) {\n          if ( ! (tf instanceof CoreLabelTokenFactory)) {\n            throw new IllegalArgumentException(\"PTBLexer: the invertible option requires a CoreLabelTokenFactory\");\n          }\n          prevWord = (CoreLabel) tf.makeToken(\"\", 0, 0);\n          prevWordAfter = new StringBuilder();\n        }\n        if (tokenizePerLine) {\n          yybegin(YyTokenizePerLine);\n        } else {\n          yybegin(YyNotTokenizePerLine);\n        }\n      }\n\n\n      /** Turn on to find out how things were tokenized. */","sourceCodeStart":61158,"sourceCodeEnd":61194,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/process/PTBLexer.java#L61158-L61194","documentation":"The PTBLexer constructor iterates over option key/value pairs and recognizes a fixed set of keys (americanize, normalizeCurrency, quotes, ellipses, dashes, untokenizable, strictTreebank3, strictFraction, strictAcronym, etc.). A key matching none of the branches hits the final else and throws this IllegalArgumentException naming the unknown key.","triggerScenarios":"Passing an options string/map to PTBTokenizer or PTBLexer with a misspelled or unsupported key, e.g. tokenize.options=\"normaliseCurrency=true,verbose=true\".","commonSituations":"Options copied from other tokenizers (OpenNLP, spaCy-style flags); typos like \"untokenizible\"; passing tokenizer-unrelated StanfordCoreNLP properties into tokenize.options.","solutions":["Replace the key with one supported by PTBLexer (e.g. americanize, normalizeCurrency, normalizeOtherBrackets, quotes, ellipses, dashes, escapeForwardSlashAsterisk, untokenizable, splitHyphenated, splitForwardSlash, splitAssimilations, strictTreebank3, strictFraction, strictAcronym, tokenizeNLs, invertible).","Fix spelling of the option key in your tokenize.options string or properties file.","Move keys that belong to other pipeline components out of the tokenizer options."],"exampleFix":"// before\nnew PTBTokenizer<>(reader, factory, \"verbose=true,untokenizable=allKeep\");\n// after\nnew PTBTokenizer<>(reader, factory, \"untokenizable=allKeep\");","handlingStrategy":"validation","validationCode":"Set<String> known = Set.of(\"americanize\",\"normalizeCurrency\",\"normalizeParentheses\",\"normalizeOtherBrackets\",\"quotes\",\"ellipses\",\"dashes\",\"escapeForwardSlashAsterisk\",\"untokenizable\",\"splitHyphenated\",\"splitForwardSlash\",\"splitAssimilations\",\"strictTreebank3\",\"strictFraction\",\"strictAcronym\",\"tokenizeNLs\",\"invertible\");\nfor (String k : optionsMap.keySet()) if (!known.contains(k)) throw new IllegalArgumentException(\"Unknown tokenizer option: \" + k);","typeGuard":null,"tryCatchPattern":"try { new PTBTokenizer<>(reader, factory, opts); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"Invalid options key\")) { throw new ConfigurationException(\"Check tokenize.options keys\", e); } throw e; }","preventionTips":["Maintain a whitelist of tokenizer keys and validate config against it.","Strip non-tokenizer properties before building tokenize.options.","Construction-time smoke test in CI with the exact options string used in production."],"tags":["tokenizer","configuration","unknown-option"],"backgroundTag":"invalid-argument-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}