apache/beam · error · ValueError

The given pcoll {pcoll_container} is not a dict, an iterable

Error message

The given pcoll {pcoll_container} is not a dict, an iterable or a PCollection.

What it means

ib.compute() shares the input-normalization logic of ib.show(): each argument must be a dict, an iterable of PCollections, a PCollection, or a DeferredBase. When iter(pcoll_container) raises TypeError, the code re-raises ValueError with this message. compute() records PCollections for later materialization, so it must receive real (or deferred) PCollections up front.

Source

Thrown at sdks/python/apache_beam/runners/interactive/interactive_beam.py:1055

    options: (optional) any additional pipeline options to use to compute the
      results.
    force_compute: (optional) if True, forces recomputation rather than using
      cached PCollections.

  Returns:
    An AsyncComputationResult object if blocking is False, otherwise None.
  """
  flatten_pcolls = []
  for pcoll_container in pcolls:
    if isinstance(pcoll_container, dict):
      flatten_pcolls.extend(pcoll_container.values())
    elif isinstance(pcoll_container, (beam.pvalue.PCollection, DeferredBase)):
      flatten_pcolls.append(pcoll_container)
    else:
      try:
        flatten_pcolls.extend(iter(pcoll_container))
      except TypeError:
        raise ValueError(
            f'The given pcoll {pcoll_container} is not a dict, an iterable or '
            'a PCollection.')

  pcolls_set = set()
  for pcoll in flatten_pcolls:
    if isinstance(pcoll, DeferredBase):
      pcoll, _ = deferred_df_to_pcollection(pcoll)
      watch({f'anonymous_pcollection_{id(pcoll)}': pcoll})
    assert isinstance(
        pcoll, beam.pvalue.PCollection
    ), f'{pcoll} is not an apache_beam.pvalue.PCollection.'
    pcolls_set.add(pcoll)

  if not pcolls_set:
    _LOGGER.info('No PCollections to compute.')
    return None

  pcoll_pipeline = next(iter(pcolls_set)).pipeline

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass PCollection objects (or dicts/lists of them) created on the interactive pipeline.
2. Inspect every element with isinstance(x, apache_beam.pvalue.PCollection) before calling compute.
3. Replace literal values with beam.Create sources on the same pipeline.
4. Fix any variable shadowing: re-run the cell defining the pcoll if a later cell reassigned the name.
5. For deferred DataFrames/Series ensure they come from ib.transform/beam dataframe API, not raw pandas objects.

Example fix

// before: ib.compute({'evens': evens, 'count': 5})  # 5 is not a PCollection | // after: five = pipeline | 'Five' >> beam.Create([5]); ib.compute({'evens': evens, 'count': five})

import apache_beam as beam; from apache_beam.dataframes import DeferredBase; def flatten_and_check(containers): return [c for c in containers if isinstance(c, (beam.pvalue.PCollection, DeferredBase)) or (isinstance(c, (dict, list, tuple)) and flatten_and_check(list(c.values() if isinstance(c, dict) else c)))] ; assert all ok before ib.compute

def is_compute_input(x): import apache_beam as beam; from apache_beam.dataframes import DeferredBase; return isinstance(x, (beam.pvalue.PCollection, DeferredBase, dict, list, tuple))

try: recording = ib.compute(*containers) | except ValueError as e: print('compute() invalid input:', e); print([(type(c), c) for c in containers])

When it happens

Trigger: ib.compute(42) or ib.compute(None); ib.compute({'k': pcoll, 'bad': 5}) where one dict value is a literal; ib.compute(pipeline_result); passing a generator of non-PCollection values; any attribute that isn't a PCollection due to variable shadowing.

Common situations: Notebooks migrating from ib.show to ib.compute for explicit recording control; passing the wrong dict level (a dict of dicts); variables overwritten by later non-beam assignments in a long notebook session; passing tf.Tensor or numpy arrays.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• The given pcoll %s is not a dict, an iterable or a PCollecti
• {pcoll} is not an apache_beam.pvalue.PCollection.
• All PCollections must belong to the same pipeline.
• The beam_sql magic tries to query PCollections from multiple
• cache_root GCS bucket path is invalid.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c40f4fdfed4004d4. Report an issue: GitHub.