apache/beam · error

could not unmarshal kv coder value component

Error message

could not unmarshal kv coder value component: %w

What it means

While unmarshaling a KV coder's value component, makeCoder peeks at the value's coder proto to see if it is an Iterable-backed coder. If peeking the component (by ID) fails, the error is wrapped as "could not unmarshal kv coder value component: %w".

Solutions

1. Read the wrapped cause to find which nested coder failed and why.
2. Ensure the value component's coder ID and any nested iterable component IDs exist in the coders map.
3. Re-serialize the pipeline with a matching SDK version if iterable coder payloads differ between versions.
4. Avoid hand-editing coder component IDs in saved pipeline protos.

cd, err := um.Coder(id)
if err != nil && strings.Contains(err.Error(), "kv coder value component") {
    log.Printf("fix nested value coder: %v", err)
}

When it happens

Trigger: makeCoder on a KV coder whose value component ID points to a coder that cannot be peeked/unmarshaled — missing coder definition, unresolvable nested component, or invalid spec.

Common situations: Pipeline protos where the KV value coder references an Iterable/StateBackedIterable coder that is absent or malformed; cross-version serialization mismatches around iterable coders.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• could not unmarshal KV coder from
• failed to marshal KV coder
• unable to decode array with iterable marker
• array len mismatch. decoding
• bad coder kind

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/eb6aec36b4d94642. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:223

		return coder.NewDouble(), nil

	case urnStringCoder:
		return coder.NewString(), nil

	case urnKVCoder:
		if len(components) != 2 {
			return nil, errors.Errorf("could not unmarshal KV coder from %v, want exactly 2 components but have %d", c, len(components))
		}

		key, err := b.Coder(components[0])
		if err != nil {
			return nil, err
		}

		id := components[1]
		elm, err := b.peek(id)
		if err != nil {
			return nil, errors.Errorf("could not unmarshal kv coder value component: %w", err)
		}

		switch elm.GetSpec().GetUrn() {
		case urnIterableCoder, urnStateBackedIterableCoder:
			iterElmID := elm.GetComponentCoderIds()[0]

			// TODO(https://github.com/apache/beam/issues/18032): If CoGBK with > 1 input, handle as special GBK. We expect
			// it to be encoded as CoGBK<K,LP<CoGBKList<V,W,..>>>. Remove this handling once
			// CoGBK has a first-class representation.

			// If the value is an iterable, and a special CoGBK type, then expand it to the real
			// CoGBK signature, instead of the special type.
			if ids, ok := b.isCoGBKList(iterElmID); ok {
				// CoGBK<K,V,W,..>

				values, err := b.Coders(ids)
				if err != nil {
					return nil, err

View on GitHub (pinned to 12126d8942)