{"record":{"id":"4f99fb9d7dc11ec2","repo":"apache/hadoop","slug":"stream-closed-4f99fb","errorCode":null,"errorMessage":"Stream closed.","messagePattern":"Stream closed\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSBlockOutputStream.java","lineNumber":101,"sourceCode":"   * @return the active block; null if there isn't one.\n   * @throws IOException on any failure to create\n   */\n  private synchronized OSSDataBlocks.DataBlock createBlockIfNeeded()\n      throws IOException {\n    if (activeBlock == null) {\n      blockId++;\n      activeBlock = blockFactory.create(blockId, blockSize, statistics);\n    }\n    return activeBlock;\n  }\n\n  /**\n   * Check for the filesystem being open.\n   * @throws IOException if the filesystem is closed.\n   */\n  void checkOpen() throws IOException {\n    if (closed.get()) {\n      throw new IOException(\"Stream closed.\");\n    }\n  }\n\n  /**\n   * The flush operation does not trigger an upload; that awaits\n   * the next block being full. What it does do is call {@code flush() }\n   * on the current block, leaving it to choose how to react.\n   * @throws IOException Any IO problem.\n   */\n  @Override\n  public synchronized void flush() throws IOException {\n    checkOpen();\n\n    OSSDataBlocks.DataBlock dataBlock = getActiveBlock();\n    if (dataBlock != null) {\n      dataBlock.flush();\n    }\n  }","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSBlockOutputStream.java#L83-L119","documentation":"Thrown by AliyunOSSBlockOutputStream.checkOpen(), which every write, flush, and sync entry point calls before touching the stream. The AtomicBoolean 'closed' is set in the close()/abort path, so this IOException means the block output stream was already closed (or aborted after a failed multipart upload) and the application kept writing or flushing.","triggerScenarios":"Calling write()/flush()/sync() on an FSDataOutputStream wrapping AliyunOSSBlockOutputStream after close() returned; two threads sharing one output stream where one closes it; double-closing from a finally block combined with a subsequent flush in another finally; writing after a failed multipart upload already set closed=true in the finally clause of close().","commonSituations":"MapReduce/Spark task cleanup writing trailer bytes after the committer closed the file; inconsistent close ordering in try-with-resources plus manual close; a task attempt that was aborted mid-write but whose record writer continues emitting.","solutions":["Audit the job for code paths that write or flush after close(); ensure exactly one component owns closing the OSS output stream (usually the output committer)","Remove redundant close() calls, or guard subsequent writes with a closed flag on your writer","Use try-with-resources so the stream closes once, deterministically, at scope exit"],"exampleFix":"// before\nout.write(data);\nout.close();\nout.flush(); // Stream closed.\n\n// after\ntry (FSDataOutputStream out = fs.create(path, overwrite)) {\n  out.write(data);\n}","handlingStrategy":"validation","validationCode":"// track ownership; only the owner closes\nprivate boolean closed = false;\nvoid writeSafe(FSDataOutputStream out, byte[] b) throws IOException {\n  if (closed) return;\n  out.write(b);\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (\"Stream closed.\".equals(e.getMessage())) { /* downstream already closed: log and stop writing */ } else throw e; }","preventionTips":["Use try-with-resources so the stream closes exactly once at scope exit","Ensure the output committer is the single owner of close(); never close job output streams manually","Guard writers with an application-level closed flag in multithreaded code"],"tags":["aliyun-oss","stream-lifecycle","write-after-close","hadoop-connector"],"backgroundTag":"write-after-close","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}