{"record":{"id":"2019d84997b9d976","repo":"stanfordnlp/CoreNLP","slug":"invalid-line-s","errorCode":null,"errorMessage":"invalid line '%s'","messagePattern":"invalid line '(.+?)'","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/time/ParsedGigawordReader.java","lineNumber":94,"sourceCode":"\n      private Annotation findAnnotation() {\n        if (this.reader == null) {\n          return null;\n        }\n        try {\n          String line;\n          StringBuilder doc = new StringBuilder();\n          while ((line = this.reader.readLine()) != null) {\n            doc.append(line);\n            doc.append('\\n');\n//            if(line.contains(\"<DOC id\")){\n//              log.info(line);\n//            }\n            if (line.equals(\"</DOC>\")) {\n              break;\n            }\n            if (line.contains(\"</DOC>\")) {\n              throw new RuntimeException(String.format(\"invalid line '%s'\", line));\n            }\n          }\n          if (line == null) {\n            this.reader.close();\n            this.reader = findReader();\n          }\n          String xml = doc.toString().replaceAll(\"&\", \"&amp;\");\n          if(xml == null || xml.equals(\"\")) {\n            return findAnnotation();\n          }\n\n          xml = xml.replaceAll(\"num=([0-9]+) (.*)\", \"num=\\\"$1\\\" $2\");\n          xml = xml.replaceAll(\"sid=(.*)>\", \"sid=\\\"$1\\\">\");\n          xml = xml.replaceAll(\"</SENT>\\n</DOC>\", \"</SENT>\\n</TEXT>\\n</DOC>\");\n          xml = new String(xml.getBytes(), \"UTF8\");\n          //log.info(\"This is what goes in:\\n\" + xml);\n          return toAnnotation(xml);\n        } catch (IOException e) {","sourceCodeStart":76,"sourceCodeEnd":112,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/time/ParsedGigawordReader.java#L76-L112","documentation":"findAnnotation scans gigaword document lines looking for the '</DOC>' terminator. A line that merely CONTAINS '</DOC>' but is not exactly that string means the document markup is malformed (trailing content on the closing tag), so it throws a RuntimeException with the offending line rather than silently mis-parsing the document.","triggerScenarios":"Reading a gigaword file whose '</DOC>' closing tag has trailing whitespace, characters, or CRLF remnants making line.equals(\"</DOC>\") false while line.contains(\"</DOC>\") is true.","commonSituations":"Corrupted or re-encoded gigaword archives; files edited by tools that appended spaces or converted line endings; concatenated documents leaving stray markup after the closing tag.","solutions":["Normalize the input file: strip trailing whitespace/CR characters after '</DOC>' and re-save.","Pre-process each line with line.trim() comparison logic or fix the file so the closing tag stands alone.","Re-download or re-extract the gigaword archive to rule out extraction corruption."],"exampleFix":"// before\nif (line.contains(\"</DOC>\")) {\n  throw new RuntimeException(String.format(\"invalid line '%s'\", line));\n}\n// after\nif (line.trim().equals(\"</DOC>\")) {\n  break;\n}","handlingStrategy":"validation","validationCode":"if (line.contains(\"</DOC>\") && !line.trim().equals(\"</DOC>\")) {\n  logger.warning(\"Malformed </DOC> line: \" + line);\n  line = line.trim(); // or sanitize the corpus beforehand\n}","typeGuard":null,"tryCatchPattern":"try {\n  return it.next();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"invalid line\")) {\n    logger.warning(\"Skipping malformed document boundary\");\n  }\n  throw e;\n}","preventionTips":["Sanitize gigaword files (strip CRLF and trailing whitespace) before parsing","Re-verify checksums of downloaded corpora","Add a corpus pre-validation pass that flags malformed DOC tags"],"tags":["parsing","gigaword","corpus-format","input-validation"],"backgroundTag":"invalid-argument-format","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}