{"record":{"id":"4f2f87d8dec1e522","repo":"apache/hadoop","slug":"getkey-stream-is-closed","errorCode":null,"errorMessage":"getKey() + \": Stream is closed!\"","messagePattern":"getKey\\(\\) \\+ \": Stream is closed!\"","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/AnalyticsStream.java","lineNumber":328,"sourceCode":"   * from parquet optimisations.\n   * Else, AAL will make a decision on which optimisations based on the file extension,\n   * if the file ends in .par or .parquet, then parquet specific optimisations are used.\n   *\n   * @param inputPolicy S3A's input file policy passed down when opening the file\n   * @return the AAL read policy\n   */\n  private InputPolicy mapS3AInputPolicyToAAL(S3AInputPolicy inputPolicy) {\n    switch (inputPolicy) {\n    case Sequential:\n      return InputPolicy.Sequential;\n    default:\n      return InputPolicy.None;\n    }\n  }\n\n  protected void throwIfClosed() throws IOException {\n    if (closed) {\n      throw new IOException(getKey() + \": \" + FSExceptionMessages.STREAM_IS_CLOSED);\n    }\n  }\n\n  /**\n   * Increment the bytes read counter if there is a stats instance\n   * and the number of bytes read is more than zero.\n   * @param bytesRead number of bytes read\n   */\n  private void incrementBytesRead(long bytesRead) {\n    getS3AStreamStatistics().bytesRead(bytesRead);\n    if (getContext().getStats() != null && bytesRead > 0) {\n      getContext().getStats().incrementBytesRead(bytesRead);\n    }\n  }\n}\n","sourceCodeStart":310,"sourceCodeEnd":344,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/AnalyticsStream.java#L310-L344","documentation":"AnalyticsStream.throwIfClosed() throws IOException(\"<key>: Stream is closed!\") when read(), seek(), available(), getPos(), readVectored() etc. are called after close(). The stream over the S3 Select query payload is single-use; once closed, every I/O method rejects use.","triggerScenarios":"Any read/seek on the FSDataInputStream from selectQuery() after close(); two owners closing/reading the same stream (e.g. wrapper closes, then caller reads); finally-block close followed by retry logic reading again.","commonSituations":"Manual stream management without try-with-resources; libraries wrapping the stream and closing it early (decompressors, JSON parsers); double-processing of a stream in error handlers; keeping a cached stream reference across request lifecycles.","solutions":["Use try-with-resources so exactly one owner closes the stream and nothing reads after.","If wrapping the stream, document/track close ownership (who closes: wrapper or caller - never both).","Null out or guard references after close; add a closed flag in your wrapper and check it before delegating.","Catch IOException and treat 'Stream is closed' as a lifecycle bug to fix, not a transient failure to retry."],"exampleFix":"// before\nFSDataInputStream in = fs.selectQuery(...).getInputStream();\nprocess(in);\nin.close();\nprocess(in); // IOException: Stream is closed!\n\n// after\ntry (FSDataInputStream in = fs.selectQuery(...).getInputStream()) {\n  process(in);\n}","handlingStrategy":"validation","validationCode":"// single-owner lifecycle: nothing reads after the try block closes\ntry (FSDataInputStream in = fs.selectQuery(select).getInputStream()) {\n  return consume(in);\n}","typeGuard":null,"tryCatchPattern":"try {\n  return in.read();\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().endsWith(\"Stream is closed\")) {\n    // lifecycle bug: reopen the stream instead of retrying blindly\n    try (FSDataInputStream retry = fs.selectQuery(select).getInputStream()) {\n      return consume(retry);\n    }\n  }\n  throw e;\n}","preventionTips":["Always open S3 Select result streams with try-with-resources.","Document which wrapper owns close() when decorating streams.","Never cache the stream beyond the scope that opened it."],"tags":["hadoop-aws","s3-select","stream-lifecycle","use-after-close"],"backgroundTag":"io-stream-already-closed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}