{"record":{"id":"05b4c46b83882498","repo":"apache/hadoop","slug":"stream-closed-05b4c4","errorCode":null,"errorMessage":"Stream closed","messagePattern":"Stream closed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java","lineNumber":868,"sourceCode":"    String msg = String.format(\"Failed to read from all available datanodes for file %s \"\n        + \"at position=%d after retrying.\", src, position);\n    DFSClient.LOG.error(msg);\n    for (Map.Entry<InetSocketAddress, List<IOException>> dataNodeExceptions :\n        exceptionMap.entrySet()) {\n      List<IOException> exceptions = dataNodeExceptions.getValue();\n      for (IOException ex : exceptions) {\n        msg = String.format(\"Exception when fetching file %s at position=%d at datanode %s:\", src,\n            position, dataNodeExceptions.getKey());\n        DFSClient.LOG.error(msg, ex);\n      }\n    }\n  }\n\n  protected synchronized int readWithStrategy(ReaderStrategy strategy)\n      throws IOException {\n    dfsClient.checkOpen();\n    if (closed.get()) {\n      throw new IOException(\"Stream closed\");\n    }\n\n    int len = strategy.getTargetLength();\n    CorruptedBlocks corruptedBlocks = new CorruptedBlocks();\n    // A map to record IOExceptions when fetching from each datanode. Key is the socketAddress of\n    // a datanode.\n    Map<InetSocketAddress, List<IOException>> exceptionMap = new HashMap<>();\n    failures = 0;\n\n    maybeRegisterBlockRefresh();\n\n    if (pos < getFileLength()) {\n      int retries = 2;\n      while (retries > 0) {\n        try {\n          // currentNode can be left as null if previous read had a checksum\n          // error on the same block. See HDFS-3067\n          if (pos > blockEnd || currentNode == null) {","sourceCodeStart":850,"sourceCodeEnd":886,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L850-L886","documentation":"readWithStrategy() first calls dfsClient.checkOpen() and then tests the stream's closed flag; any read attempted after close() throws this IOException. It is a pure use-after-close lifecycle bug on the caller side (or a FileSystem closed underneath the stream), not a cluster condition.","triggerScenarios":"Calling read() after close() on the same FSDataInputStream; a wrapper/decoder stream closing the underlying stream early; closing the shared FileSystem while cached streams are still being read (checkOpen also fires when the client is shut down).","commonSituations":"Two components sharing one stream where one closes it; try-with-resources nesting mistakes; frameworks caching FileSystem instances and closing them per-job instead of per-user/VM.","solutions":["Own the stream in exactly one place; close it once, after all reads finish","Use try-with-resources around the whole read lifetime, not around a producer that hands the stream out","Do not close shared FileSystem instances (FileSystem.get caches per URI); use IOUtils.closeStream in finally on streams only"],"exampleFix":"// before\nFSDataInputStream in = fs.open(path);\nreadHeader(in);\nin.close();\nreadBody(in); // throws \"Stream closed\"\n\n// after: keep the stream open for its whole use, close once\ntry (FSDataInputStream in = fs.open(path)) {\n  readHeader(in);\n  readBody(in);\n}","handlingStrategy":"validation","validationCode":"// one-owner wrapper that makes the lifecycle explicit at the call site\nfinal class GuardedReader implements Closeable {\n  private final FSDataInputStream in;\n  private volatile boolean closed;\n  GuardedReader(FileSystem fs, Path p) throws IOException { in = fs.open(p); }\n  int read(byte[] b) throws IOException {\n    if (closed) throw new IllegalStateException(\"reader closed\");\n    return in.read(b);\n  }\n  public synchronized void close() throws IOException {\n    if (!closed) { closed = true; in.close(); }\n  }\n}","typeGuard":"static boolean isUseAfterClose(IOException e) {\n  return e.getMessage() != null && e.getMessage().equals(\"Stream closed\");\n}","tryCatchPattern":"try {\n  return in.read(buf);\n} catch (IOException e) {\n  if (isUseAfterClose(e)) throw new IllegalStateException(\"bug: read after close\", e);\n  throw e;\n}","preventionTips":["Close each stream exactly once, from its single owner, after all reads complete","Use try-with-resources spanning the entire read lifetime","Never close shared FileSystem objects while cached streams are in use"],"tags":["hdfs","lifecycle","use-after-close","read"],"backgroundTag":"stream-already-closed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}