apache/beam · error · TypeError

Embeddings can only be generated on dict[str, str].Got dict

Error message

Embeddings can only be generated on dict[str, str].Got dict[str, {type(batch[0])}] instead.

What it means

The text-embedding EmbeddingsManager validates that each batched column value is str or bytes before running inference. If batch[0] is any other type (int, float, list, dict, None, etc.), it raises TypeError because the underlying text embedding models only accept string inputs.

Solutions

Cast the column values to str before the embedding transform (beam.Map(lambda d: {**d, 'col': str(d['col'])})).
Point the embedding transform at a column that actually contains text.
Pre-filter or handle None/non-string rows before applying the embedding.
If you meant to embed images or structured data, use the appropriate EmbeddingsManager (image or dataclass variants).

Example fix

// before
MLTransform().with_transform(SentenceTransformerEmbeddings(columns=['num_col']))
// after
data = data | beam.Map(lambda d: {**d, 'num_col': str(d['num_col'])})
data = data | MLTransform().with_transform(SentenceTransformerEmbeddings(columns=['num_col']))

Defensive patterns

Strategy: validation

Validate before calling

def validate_text_batch(batch):
    assert isinstance(batch[0], (str, bytes)), f'Expected str, got {type(batch[0])}'

Type guard

def is_text_column(values) -> bool:
    return all(isinstance(v, (str, bytes)) for v in values)

Try / catch

try:
    data | embedding_transform
except TypeError as e:
    if 'dict[str, str]' in str(e):
        data = data | beam.Map(cast_columns_to_str)
    else:
        raise

Prevention

Cast selected columns to str before embedding transforms.
Check dtypes of CSV/BigQuery columns feeding the pipeline.
Handle None/NaN values before embedding.
Use the right embeddings manager for the data modality.

When it happens

Trigger: Applying a text embedding transform (e.g. SentenceTransformerEmbeddings via MLTransform) to a column whose values are ints, floats, lists, or None instead of strings.

Common situations: PCollection elements are dicts of mixed types and a non-string column was selected for embedding; numeric/NaN values in a CSV column passed through; forgetting to cast a column to str before embedding.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/af211b38e60d0be9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/transforms/base.py:766

  For example, if the original mode is used with RunInference to take a
  PCollection[E] to a PCollection[P], this ModelHandler would take a
  PCollection[dict[str, E]] to a PCollection[dict[str, P]].

  _TextEmbeddingHandler will accept an EmbeddingsManager instance, which
  contains the details of the model to be loaded and the inference_fn to be
  used. The purpose of _TextEmbeddingHandler is to generate embeddings for
  text inputs using the EmbeddingsManager instance.

  If the input is not a text column, a RuntimeError will be raised.

  This is an internal class and offers no backwards compatibility guarantees.

  Args:
    embeddings_manager: An EmbeddingsManager instance.
  """
  def _validate_column_data(self, batch):
    if not isinstance(batch[0], (str, bytes)):
      raise TypeError(
          'Embeddings can only be generated on dict[str, str].'
          f'Got dict[str, {type(batch[0])}] instead.')

  def get_metrics_namespace(self) -> str:
    return (
        self._underlying.get_metrics_namespace() or
        'BeamML_TextEmbeddingHandler')


class _ImageEmbeddingHandler(_EmbeddingHandler):
  """
  A ModelHandler intended to be work on list[dict[str, Image]] inputs.

  The inputs to the model handler are expected to be a list of dicts.

  For example, if the original mode is used with RunInference to take a
  PCollection[E] to a PCollection[P], this ModelHandler would take a
  PCollection[dict[str, E]] to a PCollection[dict[str, P]].

View on GitHub (pinned to 12126d8942)