apache/beam · error

could not unmarshal length prefix coder component

Error message

could not unmarshal length prefix coder component: %w

What it means

After validating arity, makeCoder peeks the single sub-component of a length-prefix coder to determine the wrapped coder. If that peek fails, the error is wrapped as "could not unmarshal length prefix coder component: %w".

Solutions

1. Follow the wrapped cause to the actual failing nested coder.
2. Ensure the referenced component coder is present in the pipeline's coders map.
3. Align SDK versions between producer and consumer of the pipeline proto.
4. If the runner added length-prefixing, verify the self-describing payload is valid.

cd, err := um.Coder(id)
if err != nil && strings.Contains(err.Error(), "length prefix coder component") {
    log.Printf("resolve wrapped nested coder failure: %v", err)
}

When it happens

Trigger: makeCoder on a length-prefix coder whose component ID resolves to a coder proto that cannot be unmarshaled (missing definition, bad URN/payload).

Common situations: Runner-side length-prefixed coders referencing coders not shipped in the pipeline proto; cross-version payload format changes; corrupted serialized pipelines.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• could not unmarshal length prefix coder from
• could not unmarshal window coder
• array len mismatch. decoding
• bad coder kind
• bad decoding function

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/21a26c6d6a4f1a8e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:266

			// a CoGBK is done at the DataSource, since that's when we can check against the downstream nodes.
		}

		value, err := b.Coder(id)
		if err != nil {
			return nil, err
		}

		t := typex.New(typex.KVType, key.T, value.T)
		return &coder.Coder{Kind: coder.KV, T: t, Components: []*coder.Coder{key, value}}, nil

	case urnLengthPrefixCoder:
		if len(components) != 1 {
			return nil, errors.Errorf("could not unmarshal length prefix coder from %v, want a single sub component but have %d", c, len(components))
		}

		sub, err := b.peek(components[0])
		if err != nil {
			return nil, errors.Errorf("could not unmarshal length prefix coder component: %w", err)
		}

		// No payload means this coder was length prefixed by the runner
		// but is likely self describing - AKA a beam coder.
		// if len(sub.GetSpec().GetPayload()) == 0 {
		// 	return b.makeCoder(components[0], sub)
		// }
		// TODO(lostluck) 2018/10/17: Make this strict again, once dataflow can use
		// the portable pipeline model directly (BEAM-2885)
		switch u := sub.GetSpec().GetUrn(); u {
		case "", urnCustomCoder:
			var ref v1pb.CustomCoder
			if err := protox.DecodeBase64(string(sub.GetSpec().GetPayload()), &ref); err != nil {
				return nil, err
			}
			custom, err := decodeCustomCoder(&ref)
			if err != nil {
				return nil, err

View on GitHub (pinned to 12126d8942)