apache/beam · error · TypeCheckError

Input to GroupByKey must be a PCollection with elements comp

Error message

Input to GroupByKey must be a PCollection with elements compatible with KV[A, B]

What it means

The KV-wrapping DoFn preceding a GroupByKey unpacks each element into a key/value pair. If the element is not a 2-tuple (or dict-like pair) it raises TypeCheckError. This means the PCollection feeding GroupByKey does not contain KV[A, B]-compatible elements.

Source

Thrown at sdks/python/apache_beam/runners/portability/fn_api_runner/trigger_manager.py:67

from apache_beam.utils import windowed_value
from apache_beam.utils.timestamp import MIN_TIMESTAMP
from apache_beam.utils.timestamp import Timestamp
from apache_beam.utils.windowed_value import WindowedValue

_LOGGER = logging.getLogger(__name__)
_LOGGER.setLevel(logging.DEBUG)

K = typing.TypeVar('K')


class _ReifyWindows(DoFn):
  """Receives KV pairs, and wraps the values into WindowedValues."""
  def process(
      self, element, window=DoFn.WindowParam, timestamp=DoFn.TimestampParam):
    try:
      k, v = element
    except TypeError:
      raise TypeCheckError(
          'Input to GroupByKey must be a PCollection with '
          'elements compatible with KV[A, B]')

    yield (k, windowed_value.WindowedValue(v, timestamp, [window]))


class _GroupBundlesByKey(DoFn):
  def start_bundle(self):
    self.keys = defaultdict(list)

  def process(self, element):
    key, windowed_value = element
    self.keys[key].append(windowed_value)

  def finish_bundle(self):
    for k, vals in self.keys.items():
      yield windowed_value.WindowedValue((k, vals),
                                         MIN_TIMESTAMP, [GlobalWindow()])

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the transform immediately upstream of GroupByKey emits (key, value) tuples: use beam.Map(lambda x: (x['k'], x)) or beam.transforms.ptransform with beam.KV
2. Check with an assertion: add beam.Map(lambda kv: assert isinstance(kv, tuple) and len(kv)==2) before the GroupByKey in testing
3. If elements are single values, group by an index/key first (e.g. enumerate into (key, value) pairs)
4. Add a runtime type hint (with input_types or beam.PTransform type check) to catch the mismatch at pipeline construction time

Example fix

# before
pc | beam.Map(lambda x: x['value']) | beam.GroupByKey()  # elements are not KV
# after
pc | beam.Map(lambda x: (x['key'], x['value'])) | beam.GroupByKey()

def kv_ok(pcoll):
    return pcoll.element_type is None or pcoll.element_type == KV[Any, Any]
# or runtime check on sample elements
isinstance(elem, tuple) and len(elem) == 2

def is_kv(element):
    return isinstance(element, tuple) and len(element) == 2

try:
    _ = pcoll | beam.GroupByKey()
except TypeCheckError as e:
    log.error('Bad GroupByKey input: %s', e)  # upstream must emit (k, v) tuples
    raise

When it happens

Trigger: Calling pipeline | GroupByKey() (or p.value_as_dict etc.) on a PCollection whose elements are scalars, lists of the wrong arity, or custom objects without tuple semantics — e.g. after a Map that emits single values instead of (key, value) pairs.

Common situations: Forgetting to emit (key, value) tuples upstream (e.g. mapping to just values), a Map returning 3-tuples or single ints, or Python 2/3 style flat_map outputs that changed element arity.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Input to _GroupByKeyOnly must be a PCollection of windowed k
• Transform "{full_label}" was applied to the output of an obj
• Transform '{full_label}' expects a PCollection as input. Got
• Coder for the GroupByKey operation "%s" is not a key-value c
• Input to GroupByKey must be a PCollection with elements comp

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/79b3d0fb586c7760. Report an issue: GitHub.