What causes "error decoding byte" in apache/beam?

Use errors.Is(err, io.EOF) to separate clean end-of-stream from real failures.

error decoding byte

Error message

error decoding byte

What it means

DecodeByte reads exactly one raw byte; a read failure other than clean io.EOF is wrapped as 'error decoding byte'. As with DecodeBool, io.EOF passes through unwrapped for legitimate stream end. This points to truncated or failing input rather than an invalid value.

Source

Thrown at sdks/go/pkg/beam/core/graph/coder/bytes.go:43

// EncodeByte encodes a single byte.
func EncodeByte(v byte, w io.Writer) error {
	// Encoding: raw byte.
	if _, err := ioutilx.WriteUnsafe(w, []byte{v}); err != nil {
		return fmt.Errorf("error encoding byte: %v", err)
	}
	return nil
}

// DecodeByte decodes a single byte.
func DecodeByte(r io.Reader) (byte, error) {
	// Encoding: raw byte
	var b [1]byte
	if err := ioutilx.ReadNBufUnsafe(r, b[:]); err != nil {
		if err == io.EOF {
			return 0, err
		}
		return 0, errors.Wrap(err, "error decoding byte")
	}
	return b[0], nil
}

// EncodeBytes encodes a []byte with a length prefix per the beam protocol.
func EncodeBytes(v []byte, w io.Writer) error {
	// Encoding: size (varint) + raw data
	size := len(v)
	if err := EncodeVarInt((int64)(size), w); err != nil {
		return err
	}
	_, err := w.Write(v)
	return err

}

// DecodeBytes decodes a length prefixed []byte according to the beam protocol.
func DecodeBytes(r io.Reader) ([]byte, error) {

View on GitHub (pinned to 12126d8942)

Solutions

Use errors.Is(err, io.EOF) to separate clean end-of-stream from real failures.
Validate the data file/stream integrity and re-fetch or restore from a good copy if corrupted.
Check the source reader is open and healthy for the entire decode session.
If streaming over network, add retry/reconnect logic and ensure record framing is intact.

Example fix

// before: ignoring EOF vs real errors
b, err := DecodeByte(r)
if err != nil { return err }
// after
b, err := DecodeByte(r)
if err == io.EOF { return io.EOF }
if err != nil { return fmt.Errorf("stream corrupt: %w", err) }

Defensive patterns

Strategy: try-catch

Try / catch

// Go
b, err := coder.DecodeByte(r)
if err == io.EOF {
    return 0, io.EOF
}
if err != nil {
    return 0, fmt.Errorf("byte stream read failed: %w", err)
}

Prevention

Check reader liveness before long decode loops.
Use errors.Is(err, io.EOF) to distinguish end-of-stream.
Retry/restore corrupted or truncated data sources.
Frame network streams so partial reads are detectable.

When it happens

Trigger: DecodeByte(r) when the underlying reader returns a non-EOF error (reset connection, unreadable/corrupt file, short read with I/O error).

Common situations: Reading truncated serialized data files; network errors between pipeline workers; decoding from a closed or erroring reader in custom sources.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d68b72d23fc5f070. Report an issue: GitHub.