What causes "unable to decode slice iterable with size: %d" in apache/beam?

Verify the data was encoded by a compatible Beam version/coder

unable to decode slice iterable with size: %d

Error message

unable to decode slice iterable with size: %d

What it means

During decoding of a slice-typed iterable, the decoder's switch over the iterable size encoding reached its default case: the int32 size n decoded from the stream is not a value the slice decoder handles. The Beam iterable protocol encodes size as int32 and only specific values are valid per variant. This indicates a corrupt or incompatibly encoded stream rather than user input error.

Source

Thrown at sdks/go/pkg/beam/core/graph/coder/iterable.go:123

			if err != nil {
				return err
			}
			rv := reflect.MakeSlice(rt, 0, int(chunk))
			for chunk != 0 {
				rvi := reflect.MakeSlice(rt, int(chunk), int(chunk))
				if err := decodeToIterable(rvi, r, decodeToElem); err != nil {
					return err
				}
				rv = reflect.AppendSlice(rv, rvi)
				chunk, err = DecodeVarInt(r)
				if err != nil {
					return err
				}
			}
			ret.Set(rv)
			return nil
		default:
			return errors.Errorf("unable to decode slice iterable with size: %d", n)
		}
	}
}

// iterableDecoderForArray can decode from only the fixed sized and
// multi-chunk variant of the beam iterable protocol.
// Returns an error for other protocols (such as state backed).
func iterableDecoderForArray(rt reflect.Type, decodeToElem typeDecoderFieldReflect) func(reflect.Value, io.Reader) error {
	return func(ret reflect.Value, r io.Reader) error {
		// (1) Read count prefixed encoded data
		size, err := DecodeInt32(r)
		if err != nil {
			return err
		}
		n := int(size)
		if rt.Len() != n {
			return errors.Errorf("len mismatch decoding a %v: want %d got %d", rt, rt.Len(), n)
		}

View on GitHub (pinned to 12126d8942)

Solutions

Verify the data was encoded by a compatible Beam version/coder
Re-run the pipeline so data is re-encoded with the current coder
Inspect the raw stream around the failure to confirm corruption

Defensive patterns

Strategy: try-catch

Try / catch

v, err := decodeIterable(r)
if err != nil {
    if strings.Contains(err.Error(), "unable to decode slice iterable with size") {
        return nil, fmt.Errorf("incompatible iterable stream, re-encode from source: %w", err)
    }
    return nil, err
}

Prevention

Keep Beam versions consistent across write and read stages
Avoid replaying stored intermediates from much older Beam releases
Prefer re-running pipelines over replaying encoded data

When it happens

Trigger: Decoding a Beam iterable stream whose decoded int32 size n falls into the unhandled default branch of the slice iterable decoder — typically a negative sentinel this variant does not support.

Common situations: Cross-version data replay where an older pipeline wrote iterables with an encoding this decoder does not recognize; corrupted shuffle or windmill data streams.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4e5c8907a9d5e40b. Report an issue: GitHub.