{"record":{"id":"e56a8684abee787c","repo":"apache/hadoop","slug":"stream-is-closed-e56a86","errorCode":null,"errorMessage":"Stream is closed!","messagePattern":"Stream is closed!","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/S3APrefetchingInputStream.java","lineNumber":295,"sourceCode":"   * Gets the internal IO statistics.\n   *\n   * @return the internal IO statistics.\n   */\n  @Override\n  public IOStatistics getIOStatistics() {\n    if (!isClosed()) {\n      ioStatistics = inputStream.getIOStatistics();\n    }\n    return ioStatistics;\n  }\n\n  protected boolean isClosed() {\n    return inputStream == null;\n  }\n\n  protected void throwIfClosed() throws IOException {\n    if (isClosed()) {\n      throw new IOException(FSExceptionMessages.STREAM_IS_CLOSED);\n    }\n  }\n\n  // Unsupported functions.\n\n  @Override\n  public boolean seekToNewSource(long targetPos) throws IOException {\n    return false;\n  }\n\n  @Override\n  public boolean markSupported() {\n    return false;\n  }\n}\n","sourceCodeStart":277,"sourceCodeEnd":311,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/S3APrefetchingInputStream.java#L277-L311","documentation":"S3APrefetchingInputStream (fs.s3a.input.stream.type=Prefetch, or legacy fs.s3a.prefetch.enabled=true) throws IOException(FSExceptionMessages.STREAM_IS_CLOSED) from throwIfClosed() when read/seek/available/getPos is invoked after close(). Closing sets the underlying inputStream to null, so any later I/O on the prefetching stream fails with this guard.","triggerScenarios":"Calling read() or seek() after close() on an s3a FSDataInputStream backed by the prefetcher; reading in a background thread while another thread closed the stream; frameworks double-processing files where the second pass reads a closed stream.","commonSituations":"Concurrent readers sharing one open file handle where one thread finishes and closes; map tasks re-reading input splits after a task-level cleanup closed the stream; caching FileStatus/streams beyond the file's lifetime; race between idle-timeout close (if any) and a slow reader.","solutions":["Give the stream a single owner; use try-with-resources per consumer.","For shared reads, open a separate FileSystem/FSDataInputStream per reader instead of sharing one handle.","Synchronize close(): do not close while other threads may still read (CountDownLatch/executor shutdown discipline).","Treat this IOException as a lifecycle defect - log the stack with the operation and fix the ownership, do not retry."],"exampleFix":"// before: shared handle, racy close\nFSDataInputStream in = fs.open(path);\nnew Thread(() -> in.close()).start();\nint b = in.read(); // may throw: Stream is closed!\n\n// after: one stream per consumer\ntry (FSDataInputStream in = fs.open(path)) {\n  consume(in);\n}","handlingStrategy":"validation","validationCode":"// one stream per consumer; never share across threads\ntry (FSDataInputStream in = fs.open(path)) {\n  return consume(in);\n}","typeGuard":null,"tryCatchPattern":"try {\n  return in.read();\n} catch (IOException e) {\n  if (\"Stream is closed\".equals(e.getMessage())\n      || (e.getMessage() != null && e.getMessage().contains(\"Stream is closed\"))) {\n    LOG.error(\"read after close on {}\", path); // lifecycle bug: fix ownership\n  }\n  throw e;\n}","preventionTips":["Use try-with-resources; give each reader thread its own stream.","Close streams only after all reader threads have joined or been cancelled.","Add wrappers that fail fast on use-after-close with your own state tracking."],"tags":["hadoop-aws","prefetch","stream-lifecycle","use-after-close","concurrency"],"backgroundTag":"io-stream-already-closed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}