apache/beam · warning

chunk_to_dict_fn is deprecated, use embeddable_to_dict_fn

Error message

chunk_to_dict_fn is deprecated, use embeddable_to_dict_fn

What it means

The BigQuery ingestion transform in apache_beam.ml.rag.ingestion.bigquery renamed its keyword `chunk_to_dict_fn` to `embeddable_to_dict_fn`. Passing the old keyword still works (it is mapped internally) but triggers a DeprecationWarning with stacklevel=2 pointing at the caller; any other unexpected kwargs raise TypeError.

Solutions

1. Rename the keyword to `embeddable_to_dict_fn` in the constructor call.
2. Update shared examples/templates that still pass chunk_to_dict_fn.
3. Add a CI grep check to fail on `chunk_to_dict_fn=` occurrences.

Example fix

// before
ChunkToBigQuery(schema=schema, chunk_to_dict_fn=fn)
// after
ChunkToBigQuery(schema=schema, embeddable_to_dict_fn=fn)

if 'chunk_to_dict_fn' in kwargs:
    raise TypeError("Use 'embeddable_to_dict_fn' instead of deprecated 'chunk_to_dict_fn'")

When it happens

Trigger: `ChunkToBigQuery(..., chunk_to_dict_fn=my_fn, schema=...)` — any constructor call supplying the legacy `chunk_to_dict_fn` keyword.

Common situations: RAG ingestion pipelines written against earlier Beam RAG APIs; copy-pasted examples predating the rename; silent behavior change after a Beam upgrade.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

• database_config must be VectorDatabaseWriteConfig, got
• Dicom Client moved to…
• Dicom IO moved to apache_beam.io.gcp.healthcare.dicomcio
• document_field cannot be empty
• EmbeddableItem must contain dense embedding

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c25510b70e5a509a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/rag/ingestion/bigquery.py:70

    Example with custom schema:
      >>> schema_config = SchemaConfig(
      ...   schema={
      ...     'fields': [
      ...       {'name': 'id', 'type': 'STRING'},
      ...       {'name': 'embedding', 'type': 'FLOAT64', 'mode': 'REPEATED'},
      ...       {'name': 'source_url', 'type': 'STRING'}
      ...     ]
      ...   },
      ...   embeddable_to_dict_fn=lambda item: {
      ...       'id': item.id,
      ...       'embedding': item.embedding.dense_embedding,
      ...       'source_url': item.metadata.get('url')
      ...   }
      ... )
    """
    self.schema = schema
    if 'chunk_to_dict_fn' in kwargs:
      warnings.warn(
          "chunk_to_dict_fn is deprecated, use embeddable_to_dict_fn",
          DeprecationWarning,
          stacklevel=2)
      embeddable_to_dict_fn = kwargs.pop('chunk_to_dict_fn')
    if kwargs:
      raise TypeError(f"Unexpected keyword arguments: {', '.join(kwargs)}")
    if embeddable_to_dict_fn is None:
      raise TypeError("SchemaConfig requires embeddable_to_dict_fn")
    self.embeddable_to_dict_fn = embeddable_to_dict_fn


class BigQueryVectorWriterConfig(VectorDatabaseWriteConfig):
  def __init__(
      self,
      write_config: dict[str, Any],
      *,  # Force keyword arguments
      schema_config: Optional[SchemaConfig] = None):
    """Configuration for writing vectors to BigQuery using managed transforms.

View on GitHub (pinned to 12126d8942)