{"record":{"id":"b548e8837ec5a7c4","repo":"apache/beam","slug":"expected-chunk-to-contain-embedding-chunk-postgres-common","errorCode":null,"errorMessage":"Expected chunk to contain embedding. {chunk}","messagePattern":"Expected chunk to contain embedding\\. (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/rag/ingestion/postgres_common.py","lineNumber":44,"sourceCode":"\n\ndef chunk_embedding_fn(chunk: EmbeddableItem) -> str:\n  \"\"\"Convert embedding to PostgreSQL array string.\n\n    Formats dense embedding as a PostgreSQL-compatible array string.\n    Example: [1.0, 2.0] -> '{1.0,2.0}'\n\n    Args:\n        chunk: Input EmbeddableItem object.\n\n    Returns:\n        str: PostgreSQL array string representation of the embedding.\n\n    Raises:\n        ValueError: If chunk has no dense embedding.\n    \"\"\"\n  if chunk.embedding is None or chunk.embedding.dense_embedding is None:\n    raise ValueError(f'Expected chunk to contain embedding. {chunk}')\n  return '{' + ','.join(str(x) for x in chunk.embedding.dense_embedding) + '}'\n\n\n@dataclass\nclass ColumnSpec:\n  \"\"\"Mapping of EmbeddableItem fields to SQL columns for insertion.\n\n  Defines how to extract and format values from EmbeddableItems into\n  database columns, handling the full pipeline from Python value to\n  SQL insertion.\n\n  The insertion process works as follows:\n  - value_fn extracts a value from the EmbeddableItem and formats it as needed\n  - The value is stored in a NamedTuple field with the specified python_type\n  - During SQL insertion, the value is bound to a ? placeholder\n\n  Attributes:\n      column_name: The column name in the database table.","sourceCodeStart":26,"sourceCodeEnd":62,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/rag/ingestion/postgres_common.py#L26-L62","documentation":"chunk_embedding_fn in the PostgreSQL helper formats an EmbeddableItem's dense embedding as a Postgres array literal '{v1,v2,...}'. It requires chunk.embedding.dense_embedding to exist; when either chunk.embedding or dense_embedding is None it raises because a vector column cannot be written without a dense vector.","triggerScenarios":"Writing to Postgres with a spec whose value_fn is chunk_embedding_fn while a chunk's embedding (or its dense_embedding) is None — un-embedded items reaching the sink.","commonSituations":"Sink placed before the embedding transform in the Beam pipeline; embedding model returning None for some inputs; reusing a pipeline that produces only sparse embeddings.","solutions":["Run the embedding transform on all chunks before the Postgres writer.","Ensure the embedder fills dense_embedding (not just sparse_embedding).","Filter or re-embed chunks with chunk.embedding is None before writing."],"exampleFix":"// before\nrows = chunks | PostgresVectorWriter(config)  # chunks unembedded\n// after\nembedded = chunks | \"embed\" >> embedding_transform\nrows = embedded | PostgresVectorWriter(config)","handlingStrategy":"type-guard","validationCode":"if chunk.embedding is None or chunk.embedding.dense_embedding is None:\n    raise ValueError(f\"chunk missing dense embedding: {chunk.id}\")","typeGuard":"def has_dense_embedding(chunk) -> bool:\n    return chunk.embedding is not None and chunk.embedding.dense_embedding is not None","tryCatchPattern":"try:\n    vec = chunk_embedding_fn(chunk)\nexcept ValueError:\n    chunk = re_embed(chunk)\n    vec = chunk_embedding_fn(chunk)","preventionTips":["Order the pipeline: embed -> validate -> write","Add a validation DoFn asserting dense embeddings pre-sink","Fail loudly in your embedding fn rather than emitting None embeddings"],"tags":["python","postgres","rag","missing-embedding"],"backgroundTag":"null-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}