apache/hadoop · error · IOException

Error serializing {}

Error message

Error serializing {}

What it means

CsvRecordOutput (the deprecated org.apache.hadoop.record CSV serialization used by legacy Hadoop streaming/record apps) calls PrintStream.checkError() after writing each field; if the underlying PrintStream has entered the error state, it throws IOException 'Error serializing <tag>'. PrintStream swallows IOExceptions internally, so this is the only way the writer surfaces a broken output stream.

Source

Thrown at hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/record/CsvRecordOutput.java:44

import java.io.UnsupportedEncodingException;

import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;

/**
 * @deprecated Replaced by <a href="http://hadoop.apache.org/avro/">Avro</a>.
 */
@Deprecated
@InterfaceAudience.Public
@InterfaceStability.Stable
public class CsvRecordOutput implements RecordOutput {

  private PrintStream stream;
  private boolean isFirst = true;
    
  private void throwExceptionOnError(String tag) throws IOException {
    if (stream.checkError()) {
      throw new IOException("Error serializing "+tag);
    }
  }
 
  private void printCommaUnlessFirst() {
    if (!isFirst) {
      stream.print(",");
    }
    isFirst = false;
  }
    
  /** Creates a new instance of CsvRecordOutput */
  public CsvRecordOutput(OutputStream out) {
    try {
      stream = new PrintStream(out, true, "UTF-8");
    } catch (UnsupportedEncodingException ex) {
      throw new RuntimeException(ex);
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Check the health of the underlying OutputStream (is it closed? is the peer alive? is the disk full?) — the field tag is only where the error surfaced
2. Do not reuse a CsvRecordOutput after this IOException; create a new one on a fresh stream
3. Close streams deterministically (RecordWriter.close()) and handle that close's exceptions
4. Migrate off org.apache.hadoop.record entirely — it is deprecated in favor of Avro or Writable serialization

try { out.WriteString(tag, s); } catch (IOException ioe) { /* ioe.getMessage() names the tag; abort the record, close and discard the writer, surface the underlying stream problem */ }

When it happens

Trigger: Any Write* call (WriteString/WriteInt/WriteBuffer/...) after the wrapped OutputStream has failed: the sink was already closed, the disk is full, the downstream socket or pipe reader died, or the stream was constructed with a bad OutputStream.

Common situations: Writing CSV records to a socket whose peer closed first, writing to a closed file after a previous failure, or long-lived streams on full disks. The tag in the message tells you which field was being written when the failure was detected, not the root cause — the real cause must be inferred from the environment.

Related errors

• Error deserializing string.
• Error deserializing buffer.
• Illegal Unicode Codepoint {} in string.
• Invalid UTF-8 representation.
• Invalid UTF-8 byte {} at offset {} in length of {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/dd95bc726258f41a. Report an issue: GitHub.