apache/beam · error · TypeError

Expected data to be dicts, got

Error message

Expected data to be dicts, got {type(batch[0])} instead.

What it means

_dict_input_fn extracts text from batches whose elements must be dicts (beam.Row objects are converted via _asdict first). If the first element of a non-empty batch is not a dict, a TypeError is raised because column-based extraction cannot proceed.

Solutions

Convert your elements to dicts before MLTransform, e.g. beam.Map(lambda x: {'text': x}).
If emitting beam.Row, ensure elements are actual Row objects so _asdict conversion applies.
Check the transform's expected input type and use the appropriate type_adapter (MLTransform columns/type_adapter).

Example fix

// before
beam.Create(['hello', 'world']) | MLTransform(write_artifact_location=..., transforms=[...])
// after
beam.Create(['hello', 'world']) | beam.Map(lambda s: {'text': s}) | MLTransform(...)

Defensive patterns

Strategy: type-guard

Validate before calling

sample = next(iter(pcoll), None)
assert sample is None or isinstance(sample, dict) or hasattr(sample, '_asdict'), 'MLTransform needs dict/Row input'

Type guard

def is_dict_batch(batch) -> bool:
    return not batch or isinstance(batch[0], dict)

Try / catch

try:
    out = data | MLTransform(...)
except TypeError as e:
    if 'Expected data to be dicts' in str(e):
        data = data | beam.Map(lambda x: x._asdict() if hasattr(x, '_asdict') else {'text': x})
        out = data | MLTransform(...)

Prevention

Always beam.Map raw values into dicts with your expected keys before MLTransform
Emit beam.Row for structured data so _asdict conversion works
Unit-test the element type at the MLTransform boundary

When it happens

Trigger: Passing a PCollection of namedtuples without _asdict support, plain strings, lists, or dataclass instances to MLTransform without specifying columns compatible with dict/Row inputs.

Common situations: Feeding raw string documents into MLTransform; applying a transform that expects dict input to a pipeline emitting typed rows or objects; forgetting to add a beam.Map(lambda x: x._asdict()) step.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4800bc46275b43dd. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/transforms/base.py:192

  """
  Only for internal use. No backwards compatibility guarantees.
  """
  @abc.abstractmethod
  def append_transform(self, transform: BaseOperation):
    """
    Append transforms to the ProcessHandler.
    """


def _dict_input_fn(
    columns: Sequence[str], batch: Sequence[Union[dict[str, Any],
                                                  beam.Row]]) -> list[str]:
  """Extract text from specified columns in batch."""
  if batch and hasattr(batch[0], '_asdict'):
    batch = [row._asdict() if hasattr(row, '_asdict') else row for row in batch]

  if not batch or not isinstance(batch[0], dict):
    raise TypeError(
        'Expected data to be dicts, got '
        f'{type(batch[0])} instead.')
  result = []
  expected_keys = set(batch[0].keys())
  expected_columns = set(columns)
  # Process one batch item at a time
  for item in batch:
    item_keys = item.keys() if isinstance(item, dict) else set()
    if set(item_keys) != expected_keys:
      extra_keys = item_keys - expected_keys
      missing_keys = expected_keys - item_keys
      raise RuntimeError(
          f'All dicts in batch must have the same keys. '
          f'extra keys: {extra_keys}, '
          f'missing keys: {missing_keys}')
    missing_columns = expected_columns - item_keys
    if (missing_columns):
      raise RuntimeError(

View on GitHub (pinned to 12126d8942)