apache/beam · error · TypeCheckError

Input to GroupByKey must be a PCollection with elements comp

Error message

Input to GroupByKey must be a PCollection with elements compatible with KV[A, B]

What it means

GroupByKey requires its input elements to be key-value pairs. Beam's ReifyWindows DoFn attempts to unpack each element as (k, v); if the element is not a 2-tuple/KV, it raises TypeCheckError telling the user the input PCollection elements must be compatible with KV[A, B].

Source

Thrown at sdks/python/apache_beam/transforms/core.py:3467

  Processes an input PCollection consisting of key/value pairs represented as a
  tuple pair. The result is a PCollection where values having a common key are
  grouped together.  For example (a, 1), (b, 2), (a, 3) will result into
  (a, [1, 3]), (b, [2]).

  The implementation here is used only when run on the local direct runner.
  """
  def __init__(self, label=None):
    self._replaced_by_gbek = False
    self._inside_gbek = False
    super().__init__(label)

  class ReifyWindows(DoFn):
    def process(
        self, element, window=DoFn.WindowParam, timestamp=DoFn.TimestampParam):
      try:
        k, v = element
      except TypeError:
        raise TypeCheckError(
            'Input to GroupByKey must be a PCollection with '
            'elements compatible with KV[A, B]')

      return [(k, WindowedValue(v, timestamp, [window]))]

    def infer_output_type(self, input_type):
      key_type, value_type = trivial_inference.key_value_types(input_type)
      return typehints.KV[
          key_type, typehints.WindowedValue[value_type]]  # type: ignore[misc]

  def get_windowing(self, inputs):
    # Switch to the continuation trigger associated with the current trigger.
    windowing = inputs[0].windowing
    triggerfn = windowing.triggerfn.get_continuation_trigger()
    return Windowing(
        windowfn=windowing.windowfn,
        triggerfn=triggerfn,
        accumulation_mode=windowing.accumulation_mode,

View on GitHub (pinned to 12126d8942)

Solutions

1. Key the input first: pcoll | beam.Map(lambda x: (x.key, x)) before GroupByKey
2. Ensure upstream transforms emit 2-tuples / beam.KV elements
3. Add explicit type hints (input type beam.KV[K, V]) so the mismatch is caught earlier with a clearer message

Example fix

// before
(pc | beam.Map(lambda x: x.value)) | beam.GroupByKey()
// after
(pc | beam.Map(lambda x: (x.key, x.value))) | beam.GroupByKey()

from apache_beam.typehints import typehints
pcoll = pcoll | beam.Map(key_fn).with_input_types(beam.typehints.KV[K, V])

def is_kv_element(el) -> bool:
    try:
        k, v = el
        return True
    except (TypeError, ValueError):
        return False

try:
    keyed = pcoll | beam.GroupByKey()
except TypeCheckError as e:
    if 'must be a PCollection with elements compatible with KV' in str(e):
        keyed = pcoll | beam.Map(lambda x: (x['key'], x)) | beam.GroupByKey()
    else:
        raise

When it happens

Trigger: Applying beam.GroupByKey() to a PCollection of non-pair elements, e.g. plain ints, strings, or 3-tuples produced by an earlier Map without keying.

Common situations: Forgetting to add a Map(lambda x: (x['key'], x)) keying step before GroupByKey, or a schema change in upstream data that alters tuple arity.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Unable to convert objects of type %s to a PCollection
• Encountered unknown type {other!r}
• Proxy '{proxy}' has unsupported type '{type(proxy)}'
• Input should be a module object, got {str(module)} instead
• Cannot get a type descriptor for %s.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ae5576f6706d7b72. Report an issue: GitHub.