{"record":{"id":"f67eeeae084011bb","repo":"stanfordnlp/CoreNLP","slug":"unknown-special-lemma","errorCode":null,"errorMessage":"Unknown special lemma: ","messagePattern":"Unknown special lemma: ","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/simple/SentenceAlgorithms.java","lineNumber":177,"sourceCode":"          //       update the transition matrix.\n          updateExpectation.accept(coarseTag);\n          inLookahead = false;\n        } else if (expectNextLemma.contains(lemma)) {\n          // Case: we hit a valid word. Do something special.\n          switch (lemma) {\n            case \"of\":\n              // These prepositions are valid to subsume into a noun phrase.\n              // Update the transition matrix, and mark this as conditionally ok.\n              updateExpectation.accept('I');\n              inLookahead = true;\n              break;\n            case \"'s\":\n              // Possessives often denote a longer compound phrase\n              updateExpectation.accept('Z');\n              inLookahead = true;\n              break;\n            default:\n              throw new IllegalStateException(\"Unknown special lemma: \" + lemma);\n          }\n        } else {\n          // Case: We have transitioned to an 'invalid' state, and therefore the span should end.\n          if (inLookahead) {\n            // If we were in a lookahead token, ignore the last token (as per the lookahead definition)\n            spans.add(Span.fromValues(spanBegin, i - 1));\n          } else {\n            // Otherwise, add the span\n            spans.add(Span.fromValues(spanBegin, i));\n          }\n          // We may also have started a new span.\n          // Check to see if we have started a new span.\n          if (coarseTag == 'N' || coarseTag == 'V' || coarseTag == 'J' || coarseTag == 'X' || coarseTag == 'G') {\n            spanBegin = i;\n            updateExpectation.accept(coarseTag);\n          } else {\n            spanBegin = -1;\n          }","sourceCodeStart":159,"sourceCodeEnd":195,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/simple/SentenceAlgorithms.java#L159-L195","documentation":"In keyphraseSpans, special lemmas (like \"'s\") are handled by a switch with a default that throws IllegalStateException('Unknown special lemma: ' + lemma) when a lemma in the special-lemma set has no case in the FSA. This happens when the token set the code treats as special and the switch's cases drift apart.","triggerScenarios":"Calling keyphraseSpans()/keyphrases() on a sentence containing a token whose lemma is classified as 'special' but lacks a switch case — e.g. unusual punctuation or contractions beyond those handled ('s, and, etc. depending on version).","commonSituations":"Text with uncommon contractions or symbols; using a lemmatizer whose output forms differ from what SentenceAlgorithms expects (e.g. custom lemmatizer or different language model).","solutions":["Inspect sentence.lemmas() to find the offending lemma","Upgrade CoreNLP — newer versions add cases for more special lemmas","Normalize/strip unusual punctuation before running keyphrase extraction","Catch the IllegalStateException and skip that sentence"],"exampleFix":"// before\nList<String> kps = sentence.algorithms().keyphrases();\n// after\nList<String> kps;\ntry {\n  kps = sentence.algorithms().keyphrases();\n} catch (IllegalStateException e) {\n  kps = Collections.emptyList(); // unknown special lemma in this sentence\n}","handlingStrategy":"try-catch","validationCode":"// pre-scan for lemmas the FSA may not know\nboolean lemmasKnown = sentence.lemmas().stream()\n  .allMatch(l -> !l.startsWith(\"'\") || l.equals(\"'s\"));","typeGuard":null,"tryCatchPattern":"try {\n  List<String> kps = sentence.algorithms().keyphrases();\n} catch (IllegalStateException e) {\n  if (e.getMessage().startsWith(\"Unknown special lemma\")) {\n    kps = Collections.emptyList();\n  } else { throw e; }\n}","preventionTips":["Normalize contractions/punctuation before extraction","Keep CoreNLP updated so new special lemmas are covered","Wrap per-sentence keyphrase extraction in error handling for batch jobs"],"tags":["corenlp","keyphrase","lemmatization"],"backgroundTag":"invalid-enum-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}