apache/beam · error

failed to marshal CoGBK coder %v

Error message

failed to marshal CoGBK coder %v

What it means

Wrap returned by Add when marshaling a coder.CoGBK: AddMulti failed on the CoGBK's component coders. The wrapped inner error carries the actual cause (a component coder that could not be interned); the message identifies the CoGBK coder being serialized for diagnosis.

Source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:493

	case coder.KV:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal KV coder %v", c)
		}
		return b.internBuiltInCoder(urnKVCoder, comp...), nil

	case coder.Nullable:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal Nullable coder %v", c)
		}
		return b.internBuiltInCoder(urnNullableCoder, comp...), nil

	case coder.CoGBK:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal CoGBK coder %v", c)
		}
		value := comp[1]
		if len(comp) > 2 {
			// TODO(https://github.com/apache/beam/issues/18032): don't inject union coder for CoGBK.

			union := b.internBuiltInCoder(urnCoGBKList, comp[1:]...)
			value = b.internBuiltInCoder(urnLengthPrefixCoder, union)
		}

		// SDKs always provide iterableCoder to runners, but can receive StateBackedIterables in return.
		stream := b.internBuiltInCoder(urnIterableCoder, value)
		return b.internBuiltInCoder(urnKVCoder, comp[0], stream), nil

	case coder.ShardedKey:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal ShardedKey coder %v", c)
		}

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the wrapped error to find the failing component coder
2. Register custom key/value types with beam.RegisterType before beam.Init
3. Replace unencodable component types with serializable ones
4. Check the expansion output coders if this occurs after a cross-language transform

// Unwrap to find the failing CoGBK component
if err != nil && strings.Contains(err.Error(), "CoGBK") {
    log.Printf("CoGBK component failure: %v", errors.Unwrap(err))
}

When it happens

Trigger: CoderBuilder.Add on a coder.CoGBK where one of the key/value component coders fails to marshal.

Common situations: CoGBK PCollections with unregistered custom key or value types; cross-language expansions producing components the Go marshaller cannot encode.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• key coder for %v is %v, want %v
• varint too long
• %v
• unable to rewrite coder %v for state %v for transform %v in
• unknown coder used for ordered list state after re-write id:

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/184980426a154bdb. Report an issue: GitHub.