apache/beam · error

could not unmarshal length prefix coder from

Error message

could not unmarshal length prefix coder from %v, want a single sub component but have %d

What it means

makeCoder handles the length-prefix coder URN and expects exactly one sub-component coder. If the proto declares any other number of components, unmarshaling fails with this error listing the coder proto and actual component count.

Solutions

1. Ensure the length-prefix coder in the pipeline proto declares exactly one component coder ID.
2. Regenerate the pipeline with a standard Beam SDK instead of hand-crafting coders.
3. Check the runner's coder-expansion logic for bugs when it applies length-prefixing.
4. Compare with a working pipeline's proto to see the expected coder structure.

if lp := comps.GetCoders()[lpCoderID]; lp != nil && len(lp.GetComponentCoderIds()) != 1 {
    return fmt.Errorf("length-prefix coder %s must have 1 component, has %d", lpCoderID, len(lp.GetComponentCoderIds()))
}

cd, err := um.Coder(id)
if err != nil && strings.Contains(err.Error(), "want a single sub component") {
    log.Printf("malformed length-prefix coder: %v", err)
}

When it happens

Trigger: makeCoder on beam:coders:length_prefix:v1 where components != 1 — produced by malformed serialization or a non-conformant runner.

Common situations: Custom runners wrapping coders with length-prefix incorrectly; corrupted or hand-edited pipeline protos; foreign-language SDK interop emitting wrong arity.

Related errors

• could not unmarshal KV coder from
• could not unmarshal length prefix coder component
• could not unmarshal windowed value coder from
• ShardedKey coder requires exactly 1 component (key), got
• array len mismatch. decoding

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/992477e2e0f20e38. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:261

				t := typex.New(typex.CoGBKType, append([]typex.FullType{key.T}, coder.Types(values)...)...)
				return &coder.Coder{Kind: coder.CoGBK, T: t, Components: append([]*coder.Coder{key}, values...)}, nil
			}
			// It's valid to have a KV<k,Iter<v>> without being a CoGBK, and validating if we need to change to
			// a CoGBK is done at the DataSource, since that's when we can check against the downstream nodes.
		}

		value, err := b.Coder(id)
		if err != nil {
			return nil, err
		}

		t := typex.New(typex.KVType, key.T, value.T)
		return &coder.Coder{Kind: coder.KV, T: t, Components: []*coder.Coder{key, value}}, nil

	case urnLengthPrefixCoder:
		if len(components) != 1 {
			return nil, errors.Errorf("could not unmarshal length prefix coder from %v, want a single sub component but have %d", c, len(components))
		}

		sub, err := b.peek(components[0])
		if err != nil {
			return nil, errors.Errorf("could not unmarshal length prefix coder component: %w", err)
		}

		// No payload means this coder was length prefixed by the runner
		// but is likely self describing - AKA a beam coder.
		// if len(sub.GetSpec().GetPayload()) == 0 {
		// 	return b.makeCoder(components[0], sub)
		// }
		// TODO(lostluck) 2018/10/17: Make this strict again, once dataflow can use
		// the portable pipeline model directly (BEAM-2885)
		switch u := sub.GetSpec().GetUrn(); u {
		case "", urnCustomCoder:
			var ref v1pb.CustomCoder
			if err := protox.DecodeBase64(string(sub.GetSpec().GetPayload()), &ref); err != nil {

View on GitHub (pinned to 12126d8942)