What causes "Invalid to nest composite inside ShardedKey: %v, %v" in apache/beam?

Use a leaf type as the ShardedKey component instead of a composite.

Invalid to nest composite inside ShardedKey

Error message

Invalid to nest composite inside ShardedKey: %v, %v

What it means

typex.New rejects ShardedKey types whose single component is itself a Composite (components[0].Class() == Composite). This panics with the type and components. The invariant is that a ShardedKey must wrap a concrete, non-composite type so its coder and shard-key extraction work directly.

Solutions

Use a leaf type as the ShardedKey component instead of a composite.
If the value is logically a pair, encode it as a single concrete struct or as encoded []bytes rather than a typex composite.
Replace the nesting with an explicit outer KV: KV<K, ShardedKey<V>>.

Example fix

// before
typex.New(typex.ShardedKeyType, typex.NewKV(k, v))
// after
typex.New(typex.ShardedKeyType, vType)

Defensive patterns

Strategy: validation

Validate before calling

func validShardedKeyComp(cs []typex.FullType) bool {
    return len(cs) == 1 && cs[0] != nil && cs[0].Class() != typex.Composite
}

Try / catch

func safeNew(t typex.Type, cs ...typex.FullType) (ft typex.FullType, err error) {
    defer func() {
        if r := recover(); r != nil { err = fmt.Errorf("typex.New: %v", r) }
    }()
    return typex.New(t, cs...), nil
}

Prevention

Never nest a composite class directly inside ShardedKey.
Use KV<K, ShardedKey<V>> for keyed sharded data instead of nesting.
Check Class() before composing types programmatically.

When it happens

Trigger: Calling typex.New(typex.ShardedKeyType, compositeType) where the single component's Class() is Composite, e.g. nesting a CoGBK or another composite inside ShardedKey, via New's public callers (Prefix, CoGroupByKey, CombinePerKey, CombineGlobally, Flatten, GroupByKey).

Common situations: Attempting to shard on a windowed or nested KV value; composing ShardedKey around a CoGBK type built for join translation; refactoring code that previously nested KV types (allowed) into ShardedKey (not allowed).

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4c56e2d84516697c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/typex/fulltype.go:156

				panic("Invalid to nest WindowedValue")
			}
			return &tree{class, t, components}
		case CoGBKType:
			if len(components) < 2 {
				panic(fmt.Sprintf("Invalid number of components for CoGBK: %v", t))
			}
			if isAnyNonKVComposite(components) {
				panic(fmt.Sprintf("Invalid to nest composites inside CoGBK: %v", t))
			}
			return &tree{class, t, components}
		case TimersType:
			return &tree{class, t, components}
		case ShardedKeyType:
			if len(components) != 1 {
				panic(fmt.Sprintf("Invalid number of components for ShardedKey: %v, %v", t, components))
			}
			if components[0].Class() == Composite {
				panic(fmt.Sprintf("Invalid to nest composite inside ShardedKey: %v, %v", t, components))
			}
			return &tree{class, t, components}
		default:
			panic(fmt.Sprintf("Unexpected composite type: %v", t))
		}
	default:
		panic(fmt.Sprintf("Invalid underlying type: %v", t))
	}
}

// NOTE(herohde) 1/26/2018: we allow nested KV types and coders to support the
// CoGBK translation (using KV<K,KV<int,[]byte>> to encode keyed raw union values)
// and potentially other uses. We do not have a reasonable way to emit nested KV
// values, so user functions are still limited to non-nested KVs. Universally-typed
// KV-values might be simple to allow, for example.

func isAnyNonKVComposite(list []FullType) bool {
	for _, t := range list {

View on GitHub (pinned to 12126d8942)