{"record":{"id":"75755b41079ad057","repo":"apache/hadoop","slug":"stream-closed-75755b","errorCode":null,"errorMessage":"Stream closed","messagePattern":"Stream closed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java","lineNumber":124,"sourceCode":"  protected abstract URL getResolvedUrl(final HttpURLConnection connection\n  ) throws IOException;\n\n  @VisibleForTesting\n  protected InputStream getInputStream() throws IOException {\n    switch (status) {\n    case NORMAL:\n      break;\n    case SEEK:\n      if (in != null) {\n        in.close();\n      }\n      InputStreamAndFileLength fin = openInputStream(startPos);\n      in = fin.in;\n      fileLength = fin.length;\n      status = StreamStatus.NORMAL;\n      break;\n    case CLOSED:\n      throw new IOException(\"Stream closed\");\n    }\n    return in;\n  }\n\n  @VisibleForTesting\n  protected InputStreamAndFileLength openInputStream(long startOffset)\n      throws IOException {\n    if (startOffset < 0) {\n      throw new EOFException(\"Negative Position\");\n    }\n    // Use the original url if no resolved url exists, eg. if\n    // it's the first time a request is made.\n    final boolean resolved = resolvedURL.getURL() != null;\n    final URLOpener opener = resolved? resolvedURL: originalURL;\n\n    final HttpURLConnection connection = opener.connect(startOffset, resolved);\n    resolvedURL.setURL(getResolvedUrl(connection));\n","sourceCodeStart":106,"sourceCodeEnd":142,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java#L106-L142","documentation":"ByteRangeInputStream (the base of WebHdfsFileSystem.OffsetUrlInputStream, i.e., every WebHDFS/SWebHDFS FSDataInputStream over HTTP) is a state machine with StreamStatus NORMAL/SEEK/CLOSED. close() sets status=CLOSED; any later read()/available() calls getInputStream(), which throws this IOException. It is the HTTP equivalent of reading a stream after close() — the object cannot be reopened, you must call FileSystem.open() again.","triggerScenarios":"Call read(), read(byte[],int,int), or available() on a WebHDFS FSDataInputStream after close() — e.g., reading in a finally block after a try-with-resources block already closed it, or keeping a cached stream handle past its consumer's lifetime.","commonSituations":"try-with-resources that closes the stream while a helper (checksum computation, logging, metrics) still reads from it; sharing one open stream across components where one closes it; double-ownership of streams in wrappers. close() itself is safe to call twice — only subsequent reads fail.","solutions":["Move all reads inside the stream's owning scope (try-with-resources block) so nothing reads after close","If the stream is shared, give it exactly one owner responsible for close, or wrap it in an uncloseable filter (CloseShieldInputStream) for secondary readers","Reopen with fs.open(path) when data is still needed after close"],"exampleFix":"// before\ntry (FSDataInputStream in = fs.open(path)) {\n  in.read(buf);\n}\nlong skipped = in.read(); // IOException: Stream closed\n\n// after\ntry (FSDataInputStream in = fs.open(path)) {\n  in.read(buf);\n  long extra = in.read(); // stays inside the owning scope\n}","handlingStrategy":"validation","validationCode":"// wrap shared streams so secondary readers cannot close the primary\nInputStream shared = org.apache.commons.io.input.CloseShieldInputStream.wrap(fs.open(path));","typeGuard":null,"tryCatchPattern":"try {\n  int b = in.read();\n} catch (IOException e) {\n  if (\"Stream closed\".equals(e.getMessage())) {\n    // lifecycle bug in caller code: reopen, do not retry the dead handle\n    try (FSDataInputStream fresh = fs.open(path)) { /* reread */ }\n  } else { throw e; }\n}","preventionTips":["Own every stream in exactly one try-with-resources scope","Null the reference after close so accidental reuse fails fast as NPE","Use CloseShieldInputStream for observability/metrics wrappers that must not close the underlying stream"],"tags":["hdfs","webhdfs","io","stream-lifecycle"],"backgroundTag":"stream-already-closed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}