{"record":{"id":"417bfd4b86fc332c","repo":"apache/hadoop","slug":"unable-to-write-to-output-stream","errorCode":null,"errorMessage":"Unable to write to output stream.","messagePattern":"Unable to write to output stream\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java","lineNumber":99,"sourceCode":"  }\n  \n  /**\n   * Copies from one stream to another.\n   * \n   * @param in InputStrem to read from\n   * @param out OutputStream to write to\n   * @param buffSize the size of the buffer.\n   * @throws IOException raised on errors performing I/O.\n   */\n  public static void copyBytes(InputStream in, OutputStream out, int buffSize) \n    throws IOException {\n    PrintStream ps = out instanceof PrintStream ? (PrintStream)out : null;\n    byte buf[] = new byte[buffSize];\n    int bytesRead = in.read(buf);\n    while (bytesRead >= 0) {\n      out.write(buf, 0, bytesRead);\n      if ((ps != null) && ps.checkError()) {\n        throw new IOException(\"Unable to write to output stream.\");\n      }\n      bytesRead = in.read(buf);\n    }\n  }\n\n  /**\n   * Copies from one stream to another. <strong>closes the input and output streams \n   * at the end</strong>.\n   *\n   * @param in InputStrem to read from\n   * @param out OutputStream to write to\n   * @param conf the Configuration object.\n   * @throws IOException raised on errors performing I/O.\n   */\n  public static void copyBytes(InputStream in, OutputStream out, Configuration conf)\n    throws IOException {\n    copyBytes(in, out, conf.getInt(\n        IO_FILE_BUFFER_SIZE_KEY, IO_FILE_BUFFER_SIZE_DEFAULT), true);","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java#L81-L117","documentation":"Thrown by IOUtils.copyBytes(InputStream, OutputStream, int) when the destination is a PrintStream whose checkError() flag is set after a write. PrintStream swallows IOExceptions internally (it never throws them), so checkError() is the only way to detect that the underlying write failed. The message tells you bytes were copied into a stream that silently stopped accepting them.","triggerScenarios":"Calling the 3-arg IOUtils.copyBytes(in, out, buffSize) where 'out' is a PrintStream (e.g. System.out or System.err) and the consumer of that stream has failed: reader closed the pipe, disk full when redirected to a file, or the print stream was closed. The check only runs because 'out instanceof PrintStream'; regular OutputStreams throw their own IOException instead.","commonSituations":"Running a Hadoop CLI tool and piping stdout to 'head' or 'less' then quitting (broken pipe / SIGPIPE); redirecting tool output to a full disk or read-only location; dumping file contents to System.out via copyBytes and the terminal/pipe dies mid-copy.","solutions":["Identify why the PrintStream failed: if piping to a pager like 'head', that is expected broken-pipe behavior — pipe to 'cat' or consume the full output instead.","If redirecting to a file, check disk space (df -h) and write permissions of the target path.","Wrap the copyBytes call in try-catch (IOException) and stop the copy cleanly instead of letting the loop spin on a dead stream.","For programmatic bulk copies, pass a raw FSDataOutputStream/FileOutputStream rather than a PrintStream so failures surface as normal IOExceptions with a cause."],"exampleFix":"// before: failures are silent until checkError trips, no cause available\nIOUtils.copyBytes(fs.open(src), System.out, 4096);\n\n// after: use a real OutputStream so IOException carries the cause; handle broken pipe explicitly\ntry (FSDataInputStream in = fs.open(src);\n     OutputStream out = Files.newOutputStream(Paths.get(\"/tmp/out\"))) {\n  IOUtils.copyBytes(in, out, 4096, true);\n} catch (IOException e) {\n  throw new IOException(\"Copy to output failed: \" + e.getMessage(), e);\n}","handlingStrategy":"try-catch","validationCode":"if (out instanceof PrintStream && ((PrintStream) out).checkError()) {\n  throw new IOException(\"PrintStream already in error state — aborting copy\");\n}\nIOUtils.copyBytes(in, out, buffSize);","typeGuard":null,"tryCatchPattern":"try {\n  IOUtils.copyBytes(in, out, buffSize);\n} catch (IOException e) {\n  if (e.getMessage().equals(\"Unable to write to output stream.\")) {\n    // PrintStream swallowed the real cause (broken pipe / disk full) —\n    // stop producing, nothing you write will succeed\n    stopProducer();\n  } else { throw e; }\n}","preventionTips":["Use raw OutputStreams (FSDataOutputStream, FileOutputStream) for programmatic copies so failures carry a cause; reserve PrintStream for human-facing stdout.","When piping CLI output to pagers, expect broken pipe and exit quietly instead of dumping stack traces.","Check PrintStream.checkError() periodically in long copy loops instead of only at the end.","Use the 4-arg copyBytes(in, out, buffSize, close=true) so streams are cleaned up on failure."],"tags":["io","print-stream","broken-pipe","hadoop-common"],"backgroundTag":"output-stream-write-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}