{"record":{"id":"79ed0b433f9fa978","repo":"stanfordnlp/CoreNLP","slug":"java-io-ioexception","errorCode":null,"errorMessage":"java.io.IOException","messagePattern":"java\\.io\\.IOException","errorType":"exception","errorClass":"RuntimeIOException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/process/LexerTokenizer.java","lineNumber":65,"sourceCode":"   */\n  public LexerTokenizer(Lexer l) {\n    if (l == null) {\n      throw new IllegalArgumentException(\"You can't make a Tokenizer out of a null Lexer!\");\n    } else {\n      this.lexer = l;\n    }\n  }\n\n  /** Constructs a tokenizer from a {@link Lexer} and makes a {@link Reader}\n   *  the active input stream for the tokenizer.\n   */\n  public LexerTokenizer(Lexer l, Reader r) {\n    this(l);\n\n    try {\n      l.yyreset(r);\n    } catch (IOException e) {\n      throw new RuntimeIOException(e.getMessage());\n    }\n\n    getNext();\n  }\n\n\n  /**\n   * For testing only.\n   */\n  public static void main(String[] args) throws IOException {\n    Tokenizer<String> t = new LexerTokenizer(new JFlexDummyLexer((Reader) null), new BufferedReader(new FileReader(args[0])));\n    while (t.hasNext()) {\n      System.out.println(\"token \" + t.next());\n    }\n  }\n\n}\n","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/process/LexerTokenizer.java#L47-L83","documentation":"The LexerTokenizer(Lexer, Reader) constructor calls l.yyreset(r) to bind the lexer to the reader; if that throws an IOException it is rethrown as a RuntimeIOException whose message is the IOException's message (here shown as \"java.io.IOException\"). It means the lexer could not be reset onto the supplied reader — typically the reader itself was null or already broken.","triggerScenarios":"Calling new LexerTokenizer(lexer, reader) where yyreset throws — most often reader is null (NPE/IO inside yyreset) or the underlying stream of the reader is already closed or in an error state.","commonSituations":"Passing a Reader built from a closed InputStream; reusing a reader/stream that a previous tokenizer already consumed and closed; passing null reader because an earlier IO step failed.","solutions":["Ensure the Reader is non-null and open before constructing the tokenizer","Create a fresh Reader over the input for each tokenizer rather than reusing a consumed one","Do not close the underlying stream before tokenization completes","Catch RuntimeIOException around construction and inspect the cause for the underlying stream error"],"exampleFix":"// before\nReader r = new InputStreamReader(in); in.close(); // stream closed too early\nLexerTokenizer tok = new LexerTokenizer(new PTBLexer(), r);\n// after\nReader r = new BufferedReader(new InputStreamReader(in, StandardCharsets.UTF_8));\nLexerTokenizer tok = new LexerTokenizer(new PTBLexer(), r); // close r only after tokenizing","handlingStrategy":"validation","validationCode":"if (reader == null) throw new IllegalArgumentException(\"Reader is null\");\ntry {\n  reader.ready(); // throws IOException if the stream is closed/broken\n} catch (IOException e) {\n  throw new IllegalStateException(\"Reader already closed or broken\", e);\n}","typeGuard":null,"tryCatchPattern":"try {\n  LexerTokenizer tok = new LexerTokenizer(lexer, reader);\n} catch (RuntimeIOException e) {\n  throw new TokenizerInitException(\"Could not attach reader to lexer: \" + e.getMessage(), e);\n}","preventionTips":["Create a fresh Reader per tokenizer; never reuse a stream another tokenizer consumed","Close readers only after tokenization is complete (use try-with-resources around the whole tokenizing block)","Null-check and ready()-check the Reader before construction"],"tags":["io","tokenizer","reader","runtime-io-exception","stream-closed"],"backgroundTag":"file-read-failed","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}