apache/beam · error

failed to marshal ShardedKey coder

Error message

failed to marshal ShardedKey coder %v

What it means

Wraps any failure from AddMulti when marshalling the single component of a coder.ShardedKey coder. ShardedKey requires exactly one key component; component marshalling failure surfaces under this message.

Solutions

1. Read the wrapped error to identify the failing key component
2. Register the key type with beam.RegisterType before beam.Init
3. Use a serializable key type (primitives or registered structs)
4. Avoid wrapping ShardedKey in coders with unsupported components

// Unwrap to identify the failing key component
if err != nil && strings.Contains(err.Error(), "ShardedKey") {
    log.Printf("ShardedKey component failure: %v", errors.Unwrap(err))
}

When it happens

Trigger: CoderBuilder.Add on a coder.ShardedKey whose key component coder fails to marshal.

Common situations: ShardedKey PCollections over custom key types not registered with beam.RegisterType; reshuffle pipelines with exotic key types.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• bad coder kind
• bad decoding function
• bad encoding function
• bad window kind
• could not unmarshal coder from

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9f725f22f463fcf7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:510

		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal CoGBK coder %v", c)
		}
		value := comp[1]
		if len(comp) > 2 {
			// TODO(https://github.com/apache/beam/issues/18032): don't inject union coder for CoGBK.

			union := b.internBuiltInCoder(urnCoGBKList, comp[1:]...)
			value = b.internBuiltInCoder(urnLengthPrefixCoder, union)
		}

		// SDKs always provide iterableCoder to runners, but can receive StateBackedIterables in return.
		stream := b.internBuiltInCoder(urnIterableCoder, value)
		return b.internBuiltInCoder(urnKVCoder, comp[0], stream), nil

	case coder.ShardedKey:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal ShardedKey coder %v", c)
		}
		if len(comp) != 1 {
			return "", errors.Errorf("ShardedKey coder requires exactly 1 component (key), got %d", len(comp))
		}
		return b.internBuiltInCoder(urnShardedKeyCoder, comp...), nil

	case coder.WindowedValue:
		comp := []string{}
		if ids, err := b.AddMulti(c.Components); err != nil {
			return "", errors.Wrapf(err, "failed to marshal window coder %v", c)
		} else {
			comp = append(comp, ids...)
		}
		if id, err := b.AddWindowCoder(c.Window); err != nil {
			return "", errors.Wrapf(err, "failed to marshal window coder %v", c)
		} else {
			comp = append(comp, id)
		}

View on GitHub (pinned to 12126d8942)