{"record":{"id":"2ac634899264b6e4","repo":"stanfordnlp/CoreNLP","slug":"sentence-too-long-length","errorCode":null,"errorMessage":"Sentence too long: length ","messagePattern":"Sentence too long: length ","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/parser/lexparser/LexicalizedParserQuery.java","lineNumber":248,"sourceCode":"          TaggedWord tw = new TaggedWord(word.word(), ((HasTag) word).tag());\n          sentenceB.add(tw);\n        } else {\n          sentenceB.add(new Word(word.word()));\n        }\n      }\n      for (HasWord word : sentenceB) {\n        word.setWord(op.wordFunction.apply(word.word()));\n      }\n    } else {\n      sentenceB = new ArrayList<>(sentence);\n    }\n\n    if (op.testOptions.addMissingFinalPunctuation) {\n      addedPunct = addSentenceFinalPunctIfNeeded(sentenceB, length);\n    }\n    if (length > op.testOptions.maxLength) {\n      parseSkipped = true;\n      throw new UnsupportedOperationException(\"Sentence too long: length \" + length);\n    }\n    TreePrint treePrint = getTreePrint();\n    PrintWriter pwOut = op.tlpParams.pw();\n\n    //Insert the boundary symbol\n    if(sentence.get(0) instanceof CoreLabel) {\n      CoreLabel boundary = new CoreLabel();\n      boundary.setWord(Lexicon.BOUNDARY);\n      boundary.setValue(Lexicon.BOUNDARY);\n      boundary.setTag(Lexicon.BOUNDARY_TAG);\n      boundary.setIndex(sentence.size()+1);//1-based indexing used in the parser\n      sentenceB.add(boundary);\n    } else {\n      sentenceB.add(new TaggedWord(Lexicon.BOUNDARY, Lexicon.BOUNDARY_TAG));\n    }\n\n    if (Thread.interrupted()) {\n      throw new RuntimeInterruptedException();","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/parser/lexparser/LexicalizedParserQuery.java#L230-L266","documentation":"parseInternal enforces op.testOptions.maxLength: if the tokenized sentence is longer than the configured maximum, the query is marked parseSkipped and UnsupportedOperationException(\"Sentence too long: length N\") is thrown. The maximum exists because exhaustive PCFG parsing is exponential in practice on very long sentences.","triggerScenarios":"Calling parse() on a sentence whose token count exceeds maxLength (default often 40), or after raising processing of long documents without adjusting the limit.","commonSituations":"Parsing whole paragraphs or documents as one \"sentence\"; running the default max (40) on noisy text that under-tokenizes into giant sentences; speech-transcript or web text with no sentence boundaries.","solutions":["Increase the limit via testOptions: e.g. -maxLength 100 (op.testOptions.maxLength = 100) if memory/time allow","Run proper sentence splitting (DocumentPreprocessor / CoreNLP ssplit) so inputs are natural sentence lengths","Catch UnsupportedOperationException and treat the sentence as skipped (parseSkipped is set) with a fallback, e.g. dependency parse","Truncate or segment very long inputs before parsing"],"exampleFix":"// before\nOptions op = new Options(); // maxLength default 40\nLexicalizedParser lp = LexicalizedParser.getParserFromSerialisedFile(op, model);\nlp.parse(longSentence); // 120 tokens\n// after\nOptions op = new Options();\nop.testOptions.maxLength = 120;\nLexicalizedParser lp = LexicalizedParser.getParserFromSerialisedFile(op, model);","handlingStrategy":"validation","validationCode":"// Check length against the parser's configured limit before parsing\nif (sentence.size() > op.testOptions.maxLength) {\n  // skip, truncate, or segment the sentence\n}","typeGuard":null,"tryCatchPattern":"try {\n  Tree t = parser.parse(sentence);\n} catch (UnsupportedOperationException e) {\n  if (e.getMessage().startsWith(\"Sentence too long\")) return fallbackParse(sentence);\n  throw e;\n}","preventionTips":["Set maxLength explicitly for your domain (e.g. 100 for long transcripts)","Always run sentence splitting before constituency parsing","Split or summarize overly long inputs instead of raising limits unboundedly"],"tags":["parser","sentence-length","limits","java"],"backgroundTag":"value-out-of-range","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}