{"record":{"id":"89813be61747f62f","repo":"stanfordnlp/CoreNLP","slug":"eolstring-cannot-be-null","errorCode":null,"errorMessage":"eolString cannot be null","messagePattern":"eolString cannot be null","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/process/TokenizerAdapter.java","lineNumber":75,"sourceCode":"      }\n    } catch (IOException ioe) {\n      // do nothing, return null\n      return null;\n    }\n  }\n\n\n  /**\n   * Set the <code>String</code> returned when the inner tokenizer\n   * returns an end-of-line token.  This will only happen if the\n   * inner tokenizer has been set to <code>eolIsSignificant(true)</code>.\n   *\n   * @param eolString The String used to represent eol.  It is not allowed\n   *                  to be <code>null</code> (which would confuse line ends and file end)\n   */\n  public void setEolString(String eolString) {\n    if (eolString == null) {\n      throw new IllegalArgumentException(\"eolString cannot be null\");\n    }\n    this.eolString = eolString;\n  }\n\n\n  /**\n   * Say whether the <code>String</code> is the end-of-line token for\n   * this tokenizer.\n   *\n   * @param str The String being tested\n   * @return Whether it is the end-of-line token\n   */\n  public boolean isEol(String str) {\n    return eolString.equals(str);\n  }\n\n}\n","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/process/TokenizerAdapter.java#L57-L93","documentation":"TokenizerAdapter.setEolString enforces a non-null end-of-line string. A null eolString would make it impossible to distinguish line ends from end-of-file during tokenization, so null is rejected with IllegalArgumentException.","triggerScenarios":"Calling setEolString(null) on a TokenizerAdapter (e.g. one wrapping a WhitespaceTokenizer or Lexer) — often by passing an unset config variable or a null result of getProperty().","commonSituations":"Building a tokenizer from properties where the 'eolString' option is absent; wiring a null through factory code that should have defaulted to \"\\n\".","solutions":["Pass an explicit string such as \"\\n\" instead of null","Default the value: eolString = maybeNull != null ? maybeNull : \"\\n\"","If the value comes from properties, check containsKey before reading"],"exampleFix":"// before\ntok.setEolString(props.getProperty(\"eolString\"));\n// after\ntok.setEolString(props.getProperty(\"eolString\", \"\\n\"));","handlingStrategy":"validation","validationCode":"String eol = props.getProperty(\"eolString\");\nif (eol == null) eol = \"\\n\";","typeGuard":null,"tryCatchPattern":"try {\n  tokenizerAdapter.setEolString(eol);\n} catch (IllegalArgumentException e) {\n  tokenizerAdapter.setEolString(\"\\n\");\n}","preventionTips":["Never pass getProperty() results directly; always supply a default","Use \\n unless you have a specific reason otherwise","Treat null eolString as a config bug — fail fast upstream"],"tags":["java","null-check","tokenizer"],"backgroundTag":"null-argument","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}