{"record":{"id":"758305ed6c514013","repo":"apache/hadoop","slug":"uri-stream-is-closed","errorCode":null,"errorMessage":"<uri>: Stream is closed!","messagePattern":"<uri>: Stream is closed!","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java","lineNumber":593,"sourceCode":"    if (bytesRead > 0) {\n      pos += bytesRead;\n      nextReadPos += bytesRead;\n      incrementBytesRead(bytesRead);\n    } else {\n      streamReadResultNegative();\n    }\n    getS3AStreamStatistics().readOperationCompleted(len, bytesRead);\n    return bytesRead;\n  }\n\n  /**\n   * Verify that the input stream is open. Non blocking; this gives\n   * the last state of the volatile {@link #closed} field.\n   * @throws IOException if the connection is closed.\n   */\n  private void checkNotClosed() throws IOException {\n    if (closed) {\n      throw new IOException(getUri() + \": \" + FSExceptionMessages.STREAM_IS_CLOSED);\n    }\n  }\n\n  /**\n   * Close the stream.\n   * This triggers publishing of the stream statistics back to the filesystem\n   * statistics.\n   * This operation is synchronized, so that only one thread can attempt to\n   * close the connection; all later/blocked calls are no-ops.\n   * @throws IOException on any problem\n   */\n  @Override\n  public synchronized void close() throws IOException {\n    if (!closed) {\n      closed = true;\n      try {\n        stopVectoredIOOperations.set(true);\n        // close or abort the stream; blocking","sourceCodeStart":575,"sourceCodeEnd":611,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L575-L611","documentation":"Thrown by S3AInputStream.checkNotClosed(), which guards read/seek/positioned-read operations. The volatile closed flag is set by close()/abort(); because close() is synchronized and later calls are no-ops, any use of the stream after closing fails immediately with '<uri>: Stream is closed!' rather than touching released HTTP resources.","triggerScenarios":"read(), seek(), readFully(), or readVectored() on an S3AInputStream after close() or abort() was called; a reader thread racing a concurrent close(); reusing a cached FSDataInputStream that an engine has already closed or unbuffered.","commonSituations":"Reusing a cached file handle across query batches in Hive/Spark after the framework called close() or unbuffer(); closing streams in one method then reading in another; task cancellation closing inputs mid-iteration; custom InputFormats with confused stream ownership.","solutions":["Reopen the stream with fs.open(path) when data is still needed - a closed S3AInputStream cannot be revived","Audit close()/abort() call sites and thread ownership so no reader outlives the closer","If the engine calls unbuffer()/close() between batches, re-open the stream on the next read instead of reusing the handle","Wrap stream lifecycle in your own state machine/pool that invalidates entries on close"],"exampleFix":"// before\nFSDataInputStream in = cachedStream; // may have been closed by the engine\nin.read(buf);                 // IOException: <uri>: Stream is closed!\n\n// after - track open state yourself and re-open\nif (cachedStream == null || !cachedOpen) {\n  cachedStream = fs.open(path);\n  cachedOpen = true;\n}\ncachedStream.read(buf);","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  in.read(buf);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().endsWith(FSExceptionMessages.STREAM_IS_CLOSED)) {\n    in = fs.open(path); // recover by reopening the stream\n  } else {\n    throw e;\n  }\n}","preventionTips":["One owner per stream: the component that opens it closes it","Never cache FSDataInputStream across engine callbacks that may unbuffer() or close()","Track your own open/closed flag for shared streams and check it before every read"],"tags":["s3a","hadoop-aws","stream-lifecycle","ioexception","close","concurrency"],"backgroundTag":"stream-already-closed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}