apache/beam · error

failed to marshal KV coder

Error message

failed to marshal KV coder %v

What it means

Wraps any failure from AddMulti when marshalling the components of a coder.KV coder. If any KV component (key or value) fails to marshal, the whole KV coder marshalling fails with this message plus the underlying cause.

Solutions

1. Read the wrapped (inner) error to find which component failed
2. Register all custom types used in the KV with beam.RegisterType before beam.Init
3. Simplify or replace unencodable component types
4. Check the coder's components are all supported kinds

Example fix

// before
// KV<map[string]int, string> — map key type can't be registered
// after
// KV<MyKeyStruct, string> with beam.RegisterType(reflect.TypeOf((*MyKeyStruct)(nil)).Elem())

// Unwrap to find the failing component
wrapped := errors.Unwrap(err)
log.Printf("KV component failure: %v", wrapped)

When it happens

Trigger: CoderBuilder.Add on a coder.KV whose key or value component coder cannot be marshalled (e.g. unregistered custom types, unencodable component).

Common situations: PCollections of KV<customKey, T> where the custom key type wasn't registered; deeply nested coders where the root cause is in an inner component.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• bad coder kind
• bad decoding function
• bad encoding function
• bad window kind
• could not unmarshal coder from

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d8d9fff9b9c778c7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:479

				"Make sure the type was registered before calling beam.Init. For example: "+
				"beam.RegisterType(reflect.TypeOf((*TypeName)(nil)).Elem()). Some types, like maps, slices, arrays, channels, and functions cannot be registered as types.", c, c.Custom.Type)
		}
		data, err := protox.EncodeBase64(ref)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal custom coder %v", c)
		}
		inner := b.internCoder(&pipepb.Coder{
			Spec: &pipepb.FunctionSpec{
				Urn:     urnCustomCoder,
				Payload: []byte(data),
			},
		})
		return b.internBuiltInCoder(urnLengthPrefixCoder, inner), nil

	case coder.KV:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal KV coder %v", c)
		}
		return b.internBuiltInCoder(urnKVCoder, comp...), nil

	case coder.Nullable:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal Nullable coder %v", c)
		}
		return b.internBuiltInCoder(urnNullableCoder, comp...), nil

	case coder.CoGBK:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal CoGBK coder %v", c)
		}
		value := comp[1]
		if len(comp) > 2 {
			// TODO(https://github.com/apache/beam/issues/18032): don't inject union coder for CoGBK.

View on GitHub (pinned to 12126d8942)