apache/beam · error

decodeMultiChunkStream invalid chunk size: %v

Error message

decodeMultiChunkStream invalid chunk size: %v

What it means

After reading a chunk size varint, the multi-chunk decode stream only recognizes 0 (end of stream), a positive size, and -1 (state-backed iterable). Any other value is a protocol violation and produces 'decodeMultiChunkStream invalid chunk size: %v'. It indicates the byte stream has desynchronized from the expected chunk framing.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/fullvalue.go:378

		s.ret = FullValue{}
		return nil, io.EOF
	case s.chunk > 0:
		err := s.d.DecodeTo(s.r, &s.ret)
		if err != nil {
			return nil, errors.Wrap(err, "decodeStream value decode failed")
		}
		s.next++
		return &s.ret, nil
	case s.chunk == -1:
		// State Backed Iterable!
		stream, err := s.open(s.r)
		if err != nil {
			return nil, err
		}
		s.stream = stream
		return s.stream.Read()
	}
	return nil, errors.Errorf("decodeMultiChunkStream invalid chunk size: %v", s.chunk)
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Rerun the bundle; if it recurs deterministically, inspect custom coders for byte-count misreads that desynchronize framing.
2. Confirm all workers use the same SDK version as the pipeline expansion.
3. Enable runner debug logs to compare the sender's payload framing with what the reader consumed.

if strings.Contains(err.Error(), "invalid chunk size") {
    // framing desync: retry the bundle; investigate custom coders if deterministic
    return retryable
}

When it happens

Trigger: Read() falls through to the final return because s.chunk is negative (other than -1) after DecodeVarInt — i.e. the reader consumed garbage bytes as the chunk length due to a prior misread or corrupt transport data.

Common situations: A custom coder in the chain read a wrong number of bytes, shifting subsequent framing; corrupted or interleaved data-plane payloads; an SDK version mismatch producing a different wire framing than the decoder expects.

Related errors

• cannot claim work after restriction tracker returns false
• error decoding bool: %v
• invalid float encoding for: %v
• invalid varintz encoding for: %v
• invalid varuintz encoding for: %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6f653dfab19fd182. Report an issue: GitHub.