apache/beam · error

could not unmarshal CoderRef from %v, failed to decode Coder

Error message

could not unmarshal CoderRef from %v, failed to decode CoderRef "%v"

What it means

The urn-less coder's payload parsed as JSON but DecodeCoderRef rejected it — the CoderRef contents (URN or components) do not map to any known coder. Wrapped with this message so both the raw payload and cause are visible.

Source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:338

		wvc.Window.Payload = string(c.GetSpec().GetPayload())
		return wvc, nil

	case streamType:
		return nil, errors.Errorf("could not unmarshal stream type coder from %v, stream must be pair value", c)

	case "":
		// TODO(herohde) 11/27/2017: we still see CoderRefs from Dataflow. Handle that
		// case here, for now, so that the harness can use this logic.

		payload := c.GetSpec().GetPayload()

		var ref CoderRef
		if err := json.Unmarshal(payload, &ref); err != nil {
			return nil, errors.Wrapf(err, "could not unmarshal CoderRef from %v, failed to decode urn-less coder's payload \"%v\"", c, string(payload))
		}
		c, err := DecodeCoderRef(&ref)
		if err != nil {
			return nil, errors.Wrapf(err, "could not unmarshal CoderRef from %v, failed to decode CoderRef \"%v\"", c, string(payload))
		}
		return c, nil

	case urnIterableCoder, urnStateBackedIterableCoder:
		if len(components) != 1 {
			return nil, errors.Errorf("could not unmarshal iterable coder from %v, expected one component but got %d", c, len(components))
		}
		elm, err := b.Coder(components[0])
		if err != nil {
			return nil, err
		}
		return coder.NewI(elm), nil
	case urnTimerCoder:
		if len(components) != 2 {
			return nil, errors.Errorf("could not unmarshal timer coder from %v, expected two component but got %d", c, len(components))
		}
		elm, err := b.Coder(components[0])
		if err != nil {

View on GitHub (pinned to 12126d8942)

Solutions

1. Check the wrapped cause from DecodeCoderRef for the unknown URN
2. Register custom coder URNs via coder.CustomCoder and graphx custom coder handling on both ends
3. Align SDK versions between graph producer and decoder
4. Fix the CoderRef JSON payload fields (urn, components)

Example fix

// before: CoderRef with unknown urn "my:coder:v9"
// after: use a supported urn or register a custom coder, e.g. {"urn":"beam:coder:bytes:v1","components":[]}

// check URN is one DecodeCoderRef understands before decoding
switch ref.Urn {
case "", "beam:coder:bytes:v1", "beam:coder:kv:v1", "beam:coder:iterable:v1":
    // ok
default:
    return fmt.Errorf("unsupported coder URN %q", ref.Urn)
}

c, err := graphx.Coder(model, ref)
if err != nil {
    return fmt.Errorf("unsupported CoderRef %q: %w", payload, err)
}

When it happens

Trigger: json.Unmarshal of the payload succeeded but the CoderRef contains an unsupported/unknown URN or malformed component list passed to DecodeCoderRef.

Common situations: CoderRefs referencing custom coders not registered on the decoding side; SDK version skew where newer coder URNs are unknown; typos in hand-written CoderRef JSON.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• could not unmarshal stream type coder from %v, stream must b
• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1ed7839f914d6628. Report an issue: GitHub.