apache/beam · error

could not unmarshal sharded_key coder from

Error message

could not unmarshal sharded_key coder from %v, expected one component (key) but got %d

What it means

Error returned by makeCoder when a sharded_key coder URN has a component count other than one. A sharded-key coder wraps exactly the key coder; any other arity means the serialized coder reference is malformed or comes from an incompatible Beam version.

Solutions

1. Ensure the sharded_key coder has exactly one key coder component
2. Align SDK versions between the job submitter and the runner/harness
3. Regenerate the pipeline graph with the Go SDK
4. Fix hand-written coder JSON to include the single key component

Example fix

// before: sharded_key coder with 2 components
// after: {"urn":"beam:coder:sharded_key:v1","components":[{"urn":"beam:coder:bytes:v1","components":[]}]}

if ref.Urn == "beam:coder:sharded_key:v1" && len(ref.Components) != 1 {
    return fmt.Errorf("sharded_key coder must have exactly 1 key component, got %d", len(ref.Components))
}

When it happens

Trigger: Decoding a sharded_key coder whose components list is not length 1 — typically produced by SDK/runner version skew or a malformed graph.

Common situations: Pipelines using SDF/state+timers or runner sharded-key optimizations decoded by a harness of a different version; hand-assembled coder JSON.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• could not unmarshal iterable coder from
• could not unmarshal nullable coder from
• array len mismatch. decoding
• capacity of cache cannot be negative, got
• coder must not be nil

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7df61027da1a5ded. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:385

			return nil, err
		}
		t, err := schema.ToType(&s)
		if err != nil {
			return nil, err
		}
		return coder.NewR(typex.New(t)), nil
	case urnNullableCoder:
		if len(components) != 1 {
			return nil, errors.Errorf("could not unmarshal nullable coder from %v, expected one component but got %d", c, len(components))
		}
		elm, err := b.Coder(components[0])
		if err != nil {
			return nil, err
		}
		return coder.NewN(elm), nil
	case urnShardedKeyCoder:
		if len(components) != 1 {
			return nil, errors.Errorf("could not unmarshal sharded_key coder from %v, expected one component (key) but got %d", c, len(components))
		}
		keyC, err := b.Coder(components[0])
		if err != nil {
			return nil, err
		}
		return coder.NewSK(keyC), nil
	case urnIntervalWindow:
		return coder.NewIntervalWindowCoder(), nil

	// Special handling for the global window coder so it can be treated as
	// a general coder. Generally window coders are not used outside of
	// specific contexts, but this enables improved testing.
	// Window types are not permitted to be fulltypes, so
	// we use assignably equivalent anonymous struct types.
	case urnGlobalWindow:
		w, err := b.WindowCoder(id)
		if err != nil {
			return nil, errors.Errorf("could not unmarshal global window coder: %w", err)

View on GitHub (pinned to 12126d8942)