What causes "decodeStream value decode failed" in apache/beam?

Check that the coder used to encode the data matches the element type expected by the consuming DoFn.

decodeStream value decode failed

Error message

decodeStream value decode failed

What it means

Read on the single-value decode stream decodes the next element from the byte reader; a decode failure is wrapped as 'decodeStream value decode failed'. io.EOF is passed through cleanly, so this error specifically means bytes were present but could not be decoded into the expected type — typically a coder mismatch or corrupt payload.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/fullvalue.go:257

		s.next++
	}
	s.r = nil
	s.d = nil
	s.ret = FullValue{}
	return nil
}

// Read produces the next value in the stream.
func (s *decodeStream) Read() (*FullValue, error) {
	if s.r == nil || s.next == s.size {
		return nil, io.EOF
	}
	err := s.d.DecodeTo(s.r, &s.ret)
	if err != nil {
		if err == io.EOF {
			return nil, io.EOF
		}
		return nil, errors.Wrap(err, "decodeStream value decode failed")
	}
	s.next++
	return &s.ret, nil
}

// singleUseMultiChunkReStream is a decode on demand restream, that can handle a multi-chunk streams.
// Can only produce a single Stream because it consumes the reader.
// Must not be used for streams that might be re-iterated, causing Open to be called twice.
type singleUseMultiChunkReStream struct {
	r *byteCountReader
	d ElementDecoder

	open func(*byteCountReader) (Stream, error)
}

// Open returns the Stream from the start of the in-memory ReStream. Returns error if called twice.
func (n *singleUseMultiChunkReStream) Open() (Stream, error) {
	if n.r == nil {

View on GitHub (pinned to 12126d8942)

Solutions

Check that the coder used to encode the data matches the element type expected by the consuming DoFn.
Rerun the affected bundle — transient corruption is retried automatically by the runner.
If a custom coder is involved, validate it round-trips sample values before use.
Ensure all SDK containers/workers are the same version to avoid coder wire-format drift.

Defensive patterns

Strategy: retry

Try / catch

if wrapped := errors.Unwrap(err); wrapped != nil && wrapped != io.EOF {
    // decode corruption: mark bundle retryable, alert if it recurs on the same key
    return fmt.Errorf("retryable decode failure: %w", wrapped)
}

Prevention

Keep element types and coders in sync between producing and consuming stages.
Test custom coders with encode/decode round-trips.
Watch for recurring failures on the same partition indicating systematic corruption.

When it happens

Trigger: Read() on the decode stream where s.d.DecodeTo(s.r, &s.ret) fails with a non-EOF error while the plan is consuming an iter side input or GBK output, e.g. bytes encoded with coder A decoded with coder B.

Common situations: Coder mismatch after changing an element type without updating coders; corrupted payload from a failed data-plane hop; custom coder bug producing invalid bytes mid-record.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ce9a53ccd292cb89. Report an issue: GitHub.