{"record":{"id":"1a82a9cbeb4ddb74","repo":"stanfordnlp/CoreNLP","slug":"parsing-of-sentence-failed-will-ignore-and-conti","errorCode":null,"errorMessage":"Parsing of sentence failed.  Will ignore and continue: ${words}","messagePattern":"Parsing of sentence failed\\.  Will ignore and continue: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/edu/stanford/nlp/pipeline/ParserAnnotator.java","lineNumber":342,"sourceCode":"                && sentence.get(CoreAnnotations.SentenceIndexAnnotation.class) != null) {\n          iw.setSentIndex(sentence.get(CoreAnnotations.SentenceIndexAnnotation.class));\n        }\n      }\n    }\n  }\n\n  private List<Tree> doOneSentence(List<ParserConstraint> constraints,\n                             List<CoreLabel> words) {\n    ParserQuery pq = parser.parserQuery();\n    pq.setConstraints(constraints);\n    pq.parse(words);\n    List<Tree> trees = Generics.newLinkedList();\n    try {\n      // Use bestParse if kBest is set to 1.\n      if (this.kBest == 1) {\n        Tree t = pq.getBestParse();\n        if (t == null) {\n          log.warn(\"Parsing of sentence failed.  \" +\n              \"Will ignore and continue: \" +\n              SentenceUtils.listToString(words));\n        } else {\n          double score = pq.getBestScore();\n          t.setScore(score % -10000.0);\n          trees.add(t);\n        }\n      } else {\n        List<ScoredObject<Tree>> scoredObjects = pq.getKBestParses(this.kBest);\n        if (scoredObjects == null || scoredObjects.size() < 1) {\n          log.warn(\"Parsing of sentence failed.  \" +\n              \"Will ignore and continue: \" +\n              SentenceUtils.listToString(words));\n        } else {\n          for (ScoredObject<Tree> so : scoredObjects) {\n            // -10000 denotes unknown words\n            Tree tree = so.object();\n            tree.setScore(so.score() % -10000.0);","sourceCodeStart":324,"sourceCodeEnd":360,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/pipeline/ParserAnnotator.java#L324-L360","documentation":"ParserAnnotator's doOneSentence requests a parse for each sentence; when kBest == 1 and the parser returns null for getBestParse(), the annotator logs this warning with the sentence text and simply skips that sentence — no parse tree is added to the annotation, so downstream keys (tree/constituency parse) will be missing for that sentence.","triggerScenarios":"The underlying parser (e.g., the LexicalizedParser/QueryResult wrapped in pq) fails to produce any parse for a sentence — commonly because the sentence exceeds the parser's maxLength configuration, or contains no parseable tokens after tokenization/filtering.","commonSituations":"Long sentences over parse maxlen (default limits cause null parses); sentences consisting only of unusual tokens/symbols; constrained parsing where constraints admit no tree; degenerate input like empty or single-garbage-token sentences.","solutions":["Increase the parse maxlen (parse.maxlen or -maxLength option) if the failing sentences are simply too long.","Log/inspect the sentence text from the warning to identify pathological inputs and filter or pre-process them (e.g., split overly long sentences).","Check tokenizer/segmenter settings so sentences are not reduced to unparseable token sequences.","If null parses are expected and acceptable, the behavior is already a safe skip; make downstream code handle sentences without a ConstituencyAnnotation."],"exampleFix":"// before: default maxlen, long sentences return null parses\nprops.setProperty(\"annotators\", \"tokenize,ssplit,pos,parse\");\n\n// after\nprops.setProperty(\"annotators\", \"tokenize,ssplit,pos,parse\");\nprops.setProperty(\"parse.maxlen\", \"100\"); // raise the length limit","handlingStrategy":"fallback","validationCode":"// Pre-filter sentences that exceed the parser's max length\nprops.setProperty(\"parse.maxlen\", \"100\");\nsentences = sentences.stream()\n    .filter(s -> s.get(TokensAnnotation.class).size() <= 100)\n    .collect(Collectors.toList());","typeGuard":null,"tryCatchPattern":"// After annotation, handle sentences with no parse tree\nfor (CoreMap sentence : doc.get(SentencesAnnotation.class)) {\n  Tree tree = sentence.get(TreeAnnotation.class);\n  if (tree == null) {\n    // sentence was skipped by ParserAnnotator; use fallback (e.g., dependency-only) logic\n  }\n}","preventionTips":["Set parse.maxlen above the longest expected sentence, or split long sentences beforehand","Check tokenization so sentences don't degrade to unparseable token sequences","After annotation, null-check TreeAnnotation per sentence — the annotator silently skips failures","Monitor how often this warning fires; a high rate points to a max length or tokenizer misconfiguration"],"tags":["nlp","parsing","annotation","fallback"],"backgroundTag":"parse-returned-null","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}