{"record":{"id":"125158c6d391f29d","repo":"apache/beam","slug":"embeddableitem-must-contain-dense-embedding","errorCode":null,"errorMessage":"EmbeddableItem must contain dense embedding","messagePattern":"EmbeddableItem must contain dense embedding","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/rag/ingestion/bigquery.py","lineNumber":142,"sourceCode":"            metadata columns.\n    \n    Raises:\n        ValueError: If write_config doesn't include table specification.\n    \"\"\"\n    if 'table' not in write_config:\n      raise ValueError(\"write_config must be provided with 'table' specified\")\n\n    self.write_config = write_config\n    self.schema_config = schema_config\n\n  def create_write_transform(self) -> beam.PTransform:\n    \"\"\"Creates transform to write to BigQuery.\"\"\"\n    return _WriteToBigQueryVectorDatabase(self)\n\n\ndef _default_embeddable_to_dict_fn(item: EmbeddableItem):\n  if item.embedding is None or item.embedding.dense_embedding is None:\n    raise ValueError(\"EmbeddableItem must contain dense embedding\")\n  return {\n      'id': item.id,\n      'embedding': item.embedding.dense_embedding,\n      'content': item.content_string,\n      'metadata': [{\n          \"key\": k, \"value\": str(v)\n      } for k, v in item.metadata.items()]\n  }\n\n\ndef _default_schema():\n  return {\n      'fields': [{\n          'name': 'id', 'type': 'STRING'\n      }, {\n          'name': 'embedding', 'type': 'FLOAT64', 'mode': 'REPEATED'\n      }, {\n          'name': 'content', 'type': 'STRING'","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/rag/ingestion/bigquery.py#L124-L160","documentation":"The default EmbeddableItem-to-BigQuery-row converter requires each item to carry a dense embedding, since the 'embedding' column stores item.embedding.dense_embedding. Items without one are rejected with this ValueError at write time.","triggerScenarios":"Using the default embeddable_to_dict_fn (SchemaConfig with no custom fn) while writing EmbeddableItems whose embedding is None or whose embedding.dense_embedding is None/empty.","commonSituations":"Pipeline omitted the embedding step before the BigQuery vector write; failed/empty embeddings from the model for some records; reusing the default fn for metadata-only writes.","solutions":["Run an embedding transform on all items before VectorDatabaseWriteTransform","Filter out items without dense embeddings, or backfill them","Provide a custom embeddable_to_dict_fn if dense embeddings are not needed"],"exampleFix":"// before\nrows = pcoll | VectorDatabaseWriteTransform(bq_config)  # items unembedded\n// after\nrows = (pcoll\n    | beam.Filter(lambda i: i.embedding and i.embedding.dense_embedding)\n    | VectorDatabaseWriteTransform(bq_config))","handlingStrategy":"validation","validationCode":"bad = [it.id for it in items if not (it.embedding and it.embedding.dense_embedding)]\nassert not bad, f'items missing dense embedding: {bad}'","typeGuard":"def has_dense(item) -> bool:\n    return bool(item.embedding and item.embedding.dense_embedding)","tryCatchPattern":"try:\n    pcoll | _WriteToBigQueryVectorDatabase(cfg)\nexcept ValueError as e:\n    if 'must contain dense embedding' in str(e): route_to_reembedding(e)\n    else: raise","preventionTips":["Embed all items before BigQuery vector writes","Provide a custom embeddable_to_dict_fn if embeddings are optional in your schema","Filter unembedded items and dead-letter them instead of failing the pipeline"],"tags":["python","apache-beam","bigquery","rag","embeddings"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}