{"record":{"id":"c64421b076d536b2","repo":"apache/hadoop","slug":"stream-closed-c64421","errorCode":null,"errorMessage":"Stream closed","messagePattern":"Stream closed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java","lineNumber":395,"sourceCode":"\n  private int getStripedBufOffset(long offsetInBlockGroup) {\n    final long stripeLen = cellSize * dataBlkNum;\n    // compute the position in the curStripeBuf based on \"pos\"\n    return (int) (offsetInBlockGroup % stripeLen);\n  }\n\n  @Override\n  public synchronized boolean seekToNewSource(long targetPos)\n      throws IOException {\n    return false;\n  }\n\n  @Override\n  protected synchronized int readWithStrategy(ReaderStrategy strategy)\n      throws IOException {\n    dfsClient.checkOpen();\n    if (closed.get()) {\n      throw new IOException(\"Stream closed\");\n    }\n\n    // Number of bytes already read into buffer.\n    int result = 0;\n    int len = strategy.getTargetLength();\n    CorruptedBlocks corruptedBlocks = new CorruptedBlocks();\n    if (pos < getFileLength()) {\n      int retries = 2;\n      boolean isRetryRead = false;\n      while (retries > 0) {\n        try {\n          if (pos > blockEnd || isRetryRead) {\n            blockSeekTo(pos);\n          }\n          int realLen = (int) Math.min(len, (blockEnd - pos + 1L));\n          synchronized (infoLock) {\n            if (locatedBlocks.isLastBlockComplete()) {\n              realLen = (int) Math.min(realLen,","sourceCodeStart":377,"sourceCodeEnd":413,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java#L377-L413","documentation":"DFSStripedInputStream.readWithStrategy(ReaderStrategy) is the engine behind every read on a striped stream (read(), read(byte[]), readFully, positioned reads). It first calls dfsClient.checkOpen() — which throws 'Filesystem closed' if the whole DFSClient is shut down — and then rejects reads on a closed stream with IOException('Stream closed'). Once close() has run, all stripe state is gone and reads cannot be served.","triggerScenarios":"Any read-family call on a closed striped input stream. Two distinct precursors: the stream itself was closed (this message), or the shared DFSClient/FileSystem was closed first (checkOpen throws 'Filesystem closed' before this line is reached).","commonSituations":"Reading after the try-with-resources scope ended; one thread closing the stream on error while a reader thread is mid-read; cached FileSystem instances closed during application shutdown; test fixtures reused across test methods.","solutions":["Reopen the file and re-seek; a closed stream cannot be resurrected.","Centralize lifecycle: close exactly once, at the outermost owner, after all readers finish (use try-with-resources or reference counting for shared readers).","Match the message to the layer: 'Filesystem closed' means the FileSystem/DFSClient was closed (fix cache shutdown ordering); 'Stream closed' means only this stream is dead (reopen the path).","Synchronize close() with in-flight reads so a reader never races the closer."],"exampleFix":"// before\nFSDataInputStream in = cache.get(path);\nint n = in.read(buf); // another component already called in.close()\n\n// after\nFSDataInputStream in = cache.get(path);\nif (in == null || consumed) {\n  in = fs.open(path); // reopen instead of reading a closed stream\n  in.seek(lastGoodPos);\n}\nint n = in.read(buf);","handlingStrategy":"try-catch","validationCode":"if (streamClosedFlag.get()) {\n  in = fs.open(path);\n  in.seek(resumePos);\n}\nint n = in.read(buf);","typeGuard":null,"tryCatchPattern":"try {\n  return in.read(buf, off, len);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"closed\")) {\n    try (FSDataInputStream fresh = fs.open(path)) {\n      fresh.seek(resumePos);\n      return fresh.read(buf, off, len);\n    }\n  }\n  throw e;\n}","preventionTips":["Distinguish the two messages: 'Filesystem closed' = whole DFSClient shut down (fix FileSystem cache lifecycle); 'Stream closed' = just this stream (reopen the path).","Keep a resumePos so a reopen can continue where the failed read started.","Synchronize close() with in-flight reads; never close from an unrelated thread without a read barrier."],"tags":["hdfs","erasure-coding","input-stream","read","stream-closed","lifecycle"],"backgroundTag":"stream-already-closed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}