decodeMultiChunkStream chunk size decoding failed

Error message

decodeMultiChunkStream chunk size decoding failed

What it means

The multi-chunk decode stream reads a varint chunk size before each chunk; failure to decode that size prefix (other than clean EOF) is wrapped as 'decodeMultiChunkStream chunk size decoding failed'. It indicates the stream is malformed or cut off exactly where a chunk-length header was expected.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/fullvalue.go:351

	// If we have a stream already, use that.
	if s.stream != nil {
		return s.stream.Read()
	}

	// If our next value is at the chunk size, then re-set the chunk and size.
	if s.next == s.chunk {
		s.chunk = 0
		s.next = 0
	}

	// We're at the start of a chunk, see if there's a next chunk.
	if s.chunk == 0 && s.next == 0 {
		chunk, err := coder.DecodeVarInt(s.r.reader)
		if err != nil {
			if err == io.EOF {
				return nil, io.EOF
			}
			return nil, errors.Wrap(err, "decodeMultiChunkStream chunk size decoding failed")
		}
		s.chunk = chunk
	}
	switch {
	case s.chunk == 0:
		// If the chunk is still 0, then we're done.
		s.r = nil
		s.d = nil
		s.ret = FullValue{}
		return nil, io.EOF
	case s.chunk > 0:
		err := s.d.DecodeTo(s.r, &s.ret)
		if err != nil {
			return nil, errors.Wrap(err, "decodeStream value decode failed")
		}
		s.next++
		return &s.ret, nil
	case s.chunk == -1:

View on GitHub (pinned to 12126d8942)

Solutions

Rerun/retry the bundle — the runner will restart the stream from the sender.
Check earlier errors in the log; a prior custom-coder misread can desynchronize the stream.
Verify stable connectivity between runner and workers (Dataflow/Spark/Flink data plane health).

Defensive patterns

Strategy: retry

Try / catch

if errors.Is(err, io.EOF) { /* clean end */ } else if strings.Contains(err.Error(), "chunk size decoding failed") {
    return retryable // stream desync/corruption, runner should re-deliver
}

Prevention

Ensure custom coders read exactly the bytes they wrote to avoid stream desync.
Maintain stable network paths between runner and workers.
Rerun failures once before deep debugging.

When it happens

Trigger: Read() on decodeMultiChunkStream when s.chunk == 0 && s.next == 0 and coder.DecodeVarInt(s.r.reader) returns a non-EOF error: the underlying reader produced invalid bytes or the connection died mid-header.

Common situations: Network interruption between runners killing the gRPC data stream; desynchronized reader after an earlier partial decode (custom coder read the wrong number of bytes); runner-issued payload truncation.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/aecf331b46447624. Report an issue: GitHub.