apache/beam · error

unable to decode array with iterable marker %v

Error message

unable to decode array with iterable marker %v

What it means

After reading the element count n, the array decoder handles only the cases n >= 0 (fixed data) and n == -1 (iterable-backed / liquid-shard marker). Any other negative count is an unrecognized iterable marker and yields this error. It indicates a malformed or unsupported encoded representation for arrays.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/coder.go:825

		return errors.Errorf("array len mismatch. decoding %v but only have %v elements.", c.t, n)
	}
	switch {
	case n >= 0:
		rv := reflect.New(c.t).Elem()
		var e FullValue
		for i := 0; i < int(n); i++ {
			err := c.dec.DecodeTo(r, &e)
			if err != nil {
				return err
			}
			if e.Elm != nil {
				rv.Index(i).Set(reflect.ValueOf(e.Elm))
			}
		}
		*fv = FullValue{Elm: rv.Interface()}
		return nil
	default:
		return errors.Errorf("unable to decode array with iterable marker %v", n)
	}
}

func (c *arrayDecoder) Decode(r io.Reader) (*FullValue, error) {
	fv := &FullValue{}
	if err := c.DecodeTo(r, fv); err != nil {
		return nil, err
	}
	return fv, nil
}

type windowedValueEncoder struct {
	elm ElementEncoder
	win WindowEncoder
	// need to add pane encoder here
}

func (e *windowedValueEncoder) Encode(val *FullValue, w io.Writer) error {

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify writer and reader use compatible SDK/coder versions so iterable markers agree.
2. Check data integrity — unexpected negative counts usually indicate truncation or corruption.
3. Inspect the encoder side (encodeRR/iterable markers) to confirm only -1 is used as the iterable marker.
4. Re-encode the dataset with the current SDK version if the wire format changed.

// pre-validate an element count read from the wire
func validArrayMarker(n int) bool {
    return n >= 0 || n == -1 // -1 is the only supported iterable marker
}

if err := dec.DecodeTo(r, fv); err != nil {
    if strings.Contains(err.Error(), "iterable marker") {
        return fmt.Errorf("unsupported/corrupt iterable marker from writer: %w", err)
    }
    return err
}

When it happens

Trigger: arrayDecoder.DecodeTo encounters a negative n that is not -1 — typically corrupted data, or a producer using a different/newer marker convention than this decoder understands.

Common situations: Version skew between writer and reader SDKs with different iterable marker values; corrupted streams; manually crafted or wrongly framed encoded data.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• array len mismatch. decoding %v but only have %v elements.
• decodeStream value decode failed
• Error decoding input stream with coder {coder} in step {step
• error decoding bool: %v
• GroupIntoBatches: key coder for type %v is not deterministic

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1b8352d3a8538f67. Report an issue: GitHub.