GroupIntoBatchesWithShardedKey: type parameter K (%v) does n

Error message

GroupIntoBatchesWithShardedKey: type parameter K (%v) does not match input key type (%v)

What it means

GroupIntoBatchesWithShardedKey is generic over key type K and checks that the input PCollection's KV key component equals K via reflect.TypeOf(zero K). A mismatch means the wrapped KV pairs cannot be typed as K, so the transform panics with both type names in the message. Align K with the input key type or convert the keys.

Source

Thrown at sdks/go/pkg/beam/transforms/batch/batch.go:670

//
// Sharding spreads the processing of a single hot logical key across
// multiple workers: each shard is independent state, so distributed
// runners can parallelize without the user's key type changing.
func GroupIntoBatchesWithShardedKey[K any](s beam.Scope, params Params, col beam.PCollection) beam.PCollection {
	s = s.Scope("batch.GroupIntoBatchesWithShardedKey")

	if err := params.validate(); err != nil {
		panic(fmt.Errorf("GroupIntoBatchesWithShardedKey: %w", err))
	}
	if !typex.IsKV(col.Type()) {
		panic(fmt.Errorf(
			"GroupIntoBatchesWithShardedKey: input PCollection must be KV-typed; got %v",
			col.Type()))
	}
	keyFT := col.Type().Components()[0]
	var zero K
	if keyFT.Type() != reflect.TypeOf(zero) {
		panic(fmt.Errorf(
			"GroupIntoBatchesWithShardedKey: type parameter K (%v) does not match input key type (%v)",
			reflect.TypeOf(zero), keyFT.Type()))
	}

	wrapped := beam.ParDo(s, &wrapShardedKeyFn[K]{}, col)
	return GroupIntoBatches(s, params, wrapped)
}

View on GitHub (pinned to 12126d8942)

Solutions

Set K to match the input key type (or vice versa)
Insert a ParDo that converts keys to type K before the call
Check reflect.TypeOf(zero K) against col.Type().Components()[0] before calling
Pin K explicitly at the call site rather than relying on inference

Example fix

// before
batch.GroupIntoBatchesWithShardedKey[string](s, params, intKeyedKV)
// after
batch.GroupIntoBatchesWithShardedKey[int](s, params, intKeyedKV)

Defensive patterns

Strategy: type-guard

Validate before calling

var zero K
if col.Type().Components()[0].Type() != reflect.TypeOf(zero) {
    return fmt.Errorf("K does not match input key type")
}

Type guard

func keyMatches[K any](col beam.PCollection) bool {
    var zero K
    return col.Type().Components()[0].Type() == reflect.TypeOf(zero)
}

Prevention

Pin K explicitly instead of relying on inference
Convert keys with a ParDo when upstream types change
Add unit tests asserting K matches the input key type

When it happens

Trigger: Calling GroupIntoBatchesWithShardedKey[K] where K differs from the input's KV key component, e.g. GroupIntoBatchesWithShardedKey[string] fed a KV<int, V> collection.

Common situations: Relying on Go type inference after an upstream key-type change; copy-pasting a generic call and forgetting to update K; an intermediate ParDo changing the key type silently.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9277208190b4e02a. Report an issue: GitHub.