apache/beam · error · TypeError

Embeddings can only be generated on dict[str, dataclass]…

Error message

Embeddings can only be generated on  dict[str, dataclass] types. Got dict[str, {type(batch[0])}] instead.

What it means

The dataclass-based embedding path expects column values to be instances of a dataclass whose fields describe model inputs (e.g. framework tensors per modality). Primitive values (int, str, float, bool) are rejected with TypeError, since the model wrapper can only unpack structured dataclass inputs.

Solutions

Wrap each value in the dataclass expected by the embedding implementation (the module-specific Inputs dataclass).
Convert dicts to the dataclass: MyInputs(**d) before applying the transform.
Use the text or image embeddings manager if your data is actually plain strings or images.

Example fix

// before
rows | beam.Map(lambda d: {'col': d['text']}) | dataclass_embedding
// after
from dataclasses import dataclass
@dataclass
class ColInputs:
  text: str
rows | beam.Map(lambda d: {'col': ColInputs(d['text'])}) | dataclass_embedding

Defensive patterns

Strategy: validation

Validate before calling

from dataclasses import is_dataclass
def validate_dataclass_batch(batch):
    assert is_dataclass(batch[0]), f'Expected dataclass, got {type(batch[0])}'

Type guard

def is_dataclass_column(values) -> bool:
    from dataclasses import is_dataclass
    return all(is_dataclass(v) for v in values)

Try / catch

try:
    data | dataclass_embedding
except TypeError as e:
    if 'dataclass' in str(e):
        data = data | beam.Map(lambda d: {k: ColInputs(**v) for k, v in d.items()})
    else:
        raise

Prevention

Wrap model inputs in the dataclass type expected by the embedding implementation.
Convert dicts to the dataclass before the transform.
Note dicts are not dataclasses; use the documented input type.
Test one batch locally before running the full pipeline.

When it happens

Trigger: Applying a multimodal/dataclass embedding transform to a column whose values are primitives instead of dataclass instances; feeding raw strings or numbers where a structured record was expected.

Common situations: Forgetting to wrap model inputs in the expected dataclass before the transform; passing a plain dict (dicts are not dataclass instances) instead of the required dataclass type.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bcaed1377ec69214. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/transforms/base.py:841

  _MultiModalEmbeddingHandler will accept an EmbeddingsManager instance, which
  contains the details of the model to be loaded and the inference_fn to be
  used. The purpose of _MultiMOdalEmbeddingHandler is to generate embeddings
  for image, video, and text inputs using the EmbeddingsManager instance.

  If the input is not an Image representation column, a RuntimeError will be
  raised.

  This is an internal class and offers no backwards compatibility guarantees.

  Args:
    embeddings_manager: An EmbeddingsManager instance.
  """
  def _validate_column_data(self, batch):
    # Don't want to require framework-specific imports
    # here, so just catch columns of primatives for now.
    if isinstance(batch[0], (int, str, float, bool)):
      raise TypeError(
          'Embeddings can only be generated on '
          ' dict[str, dataclass] types. '
          f'Got dict[str, {type(batch[0])}] instead.')

  def get_metrics_namespace(self) -> str:
    return (
        self._underlying.get_metrics_namespace() or
        'BeamML_MultiModalEmbeddingHandler')

View on GitHub (pinned to 12126d8942)