apache/beam · error · TypeError

SchemaConfig requires embeddable_to_dict_fn

Error message

SchemaConfig requires embeddable_to_dict_fn

What it means

SchemaConfig requires a function that converts an EmbeddableItem into a dict row matching the BigQuery schema. If embeddable_to_dict_fn is None (and the deprecated chunk_to_dict_fn was not supplied), __init__ raises this TypeError because the writer would have no way to map items to rows.

Source

Thrown at sdks/python/apache_beam/ml/rag/ingestion/bigquery.py:78

      ...   },
      ...   embeddable_to_dict_fn=lambda item: {
      ...       'id': item.id,
      ...       'embedding': item.embedding.dense_embedding,
      ...       'source_url': item.metadata.get('url')
      ...   }
      ... )
    """
    self.schema = schema
    if 'chunk_to_dict_fn' in kwargs:
      warnings.warn(
          "chunk_to_dict_fn is deprecated, use embeddable_to_dict_fn",
          DeprecationWarning,
          stacklevel=2)
      embeddable_to_dict_fn = kwargs.pop('chunk_to_dict_fn')
    if kwargs:
      raise TypeError(f"Unexpected keyword arguments: {', '.join(kwargs)}")
    if embeddable_to_dict_fn is None:
      raise TypeError("SchemaConfig requires embeddable_to_dict_fn")
    self.embeddable_to_dict_fn = embeddable_to_dict_fn


class BigQueryVectorWriterConfig(VectorDatabaseWriteConfig):
  def __init__(
      self,
      write_config: dict[str, Any],
      *,  # Force keyword arguments
      schema_config: Optional[SchemaConfig] = None):
    """Configuration for writing vectors to BigQuery using managed transforms.
    
    Supports both default schema (id, embedding, content, metadata columns) and
    custom schemas through SchemaConfig.

    Example with default schema:
      >>> config = BigQueryVectorWriterConfig(
      ...     write_config={'table': 'project.dataset.embeddings'})

View on GitHub (pinned to 12126d8942)

Solutions

  1. Pass a callable embeddable_to_dict_fn(item: EmbeddableItem) -> dict to SchemaConfig
  2. Or pass the deprecated chunk_to_dict_fn (emits a DeprecationWarning) if migrating gradually
  3. If rows match the default shape, use the module-provided _default_embeddable_to_dict_fn behavior as a template

Example fix

// before
sc = SchemaConfig()
// after
sc = SchemaConfig(embeddable_to_dict_fn=lambda item: {
    'id': item.id, 'embedding': item.embedding.dense_embedding,
    'content': item.content_string, 'metadata': []})
Defensive patterns

Strategy: validation

Validate before calling

assert callable(getattr(sc_kwargs.get('embeddable_to_dict_fn'), '__call__', None)), 'embeddable_to_dict_fn required'

Type guard

def has_dict_fn(kwargs: dict) -> bool:
    fn = kwargs.get('embeddable_to_dict_fn') or kwargs.get('chunk_to_dict_fn')
    return callable(fn)

Try / catch

try:
    sc = SchemaConfig(**kwargs)
except TypeError as e:
    if 'requires embeddable_to_dict_fn' in str(e): sc = SchemaConfig(embeddable_to_dict_fn=default_fn)
    else: raise

Prevention

When it happens

Trigger: Constructing SchemaConfig() with no embeddable_to_dict_fn argument, or passing embeddable_to_dict_fn=None explicitly.

Common situations: Omitting the function in config builders; conditional code paths that default the fn to None; migrating from chunk_to_dict_fn but misspelling the new name so both remain None.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f1218838274acb14. Report an issue: GitHub.