apache/beam · error · ValueError

Input to must be compatible with KV[Any, Any]. Found .

Error message

Input to %s must be compatible with KV[Any, Any]. Found %s.

What it means

coerce_to_kv_type cannot convert the given type hint into a KV[Any, Any] form. After checking empty, tuple, Any, and Union hints, anything else (dict, list, scalar hints, etc.) raises. Beam requires key/value-shaped input for keyed transforms.

Solutions

Emit 2-tuples upstream: pc | beam.Map(lambda d: (d['k'], d['v'])).
Add an explicit with_output_types(KV[K, V]) on the producing stage so errors surface earlier.
If input is a dict side input, iterate pvalue.AsDict(...).items() inside a DoFn instead of using keyed transforms.
Use beam.KV keys explicitly, e.g. beam.pvalue.TaggedOutput or KV hints, for clarity.

Example fix

// before
side = p | 'side' >> beam.Create([{'a': 1}])
pc | beam.CombinePerKey(sum)  # input is dict hint
// after
pc2 = side | beam.FlatMap(lambda d: d.items())
pc | beam.CombinePerKey(sum)  # input is KV tuples

Defensive patterns

Strategy: validation

Validate before calling

if not isinstance(element, tuple) or len(element) != 2:
    raise TypeError(f'expected KV 2-tuple, got {type(element).__name__}')

Type guard

def is_kv_pair(x) -> bool:
    return isinstance(x, tuple) and len(x) == 2

Try / catch

try:
    pc | beam.CombinePerKey(fn)
except ValueError:
    pc = pc | beam.Map(lambda x: (x[0], x[1:])) | beam.CombinePerKey(fn)

Prevention

Convert dict side inputs to items() before keyed transforms
Annotate PCollection output types explicitly with KV[K, V]
Keep a single DoFn responsible for producing KV-shaped output

When it happens

Trigger: Feeding a PCollection hinted as Dict[K, V], List[V], str, or int into GroupByKey / CombinePerKey / CoGroupByKey, or calling coerce_to_kv_type directly with such a hint.

Common situations: Treating a dict side input as KV pairs for CombinePerKey; forgetting to emit key/value tuples upstream; hint mismatch after refactoring a DoFn's output type.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6d2993471470353e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/typehints/typehints.py:1638

    return KV[Any, Any]
  elif isinstance(element_type, TupleHint.TupleConstraint):
    if len(element_type.tuple_types) == 2:
      return element_type
    else:
      raise ValueError(
          "Tuple input to %s must have two components. "
          "Found %s." % (consumer, element_type))
  elif isinstance(element_type, AnyTypeConstraint):
    # `Any` type needs to be replaced with a KV[Any, Any] to
    # satisfy the KV form.
    return KV[Any, Any]
  elif isinstance(element_type, UnionConstraint):
    union_types = [coerce_to_kv_type(t) for t in element_type.union_types]
    return KV[Union[tuple(t.tuple_types[0] for t in union_types)],
              Union[tuple(t.tuple_types[1] for t in union_types)]]
  else:
    # TODO: Possibly handle other valid types.
    raise ValueError(
        "Input to %s must be compatible with KV[Any, Any]. "
        "Found %s." % (consumer, element_type))

View on GitHub (pinned to 12126d8942)