{"record":{"id":"5af7f5834d7a367e","repo":"apache/beam","slug":"expected-chunk-to-contain-embedding-chunk","errorCode":null,"errorMessage":"Expected chunk to contain embedding. {chunk}","messagePattern":"Expected chunk to contain embedding\\. (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/rag/ingestion/mysql_common.py","lineNumber":43,"sourceCode":"\n\ndef chunk_embedding_fn(chunk: EmbeddableItem) -> str:\n  \"\"\"Convert embedding to MySQL vector string format.\n\n    Formats dense embedding as a MySQL-compatible vector string.\n    Example: [1.0, 2.0] -> '[1.0,2.0]'\n\n    Args:\n        chunk: Input EmbeddableItem object.\n    \n    Returns:\n        str: MySQL vector string representation of the embedding.\n    \n    Raises:\n        ValueError: If chunk has no dense embedding.\n    \"\"\"\n  if chunk.embedding is None or chunk.embedding.dense_embedding is None:\n    raise ValueError(f'Expected chunk to contain embedding. {chunk}')\n  return '[' + ','.join(str(x) for x in chunk.embedding.dense_embedding) + ']'\n\n\n@dataclass\nclass ColumnSpec:\n  \"\"\"Mapping of EmbeddableItem fields to SQL columns for insertion.\n\n  Defines how to extract and format values from EmbeddableItems into MySQL\n  database columns, handling the full pipeline from Python value to SQL\n  insertion.\n\n  The insertion process works as follows:\n  - value_fn extracts a value from the EmbeddableItem and formats it as needed\n  - The value is stored in a NamedTuple field with the specified python_type\n  - During SQL insertion, the value is bound to a ? placeholder\n\n  Attributes:\n      column_name: The column name in the database table.","sourceCodeStart":25,"sourceCodeEnd":61,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/rag/ingestion/mysql_common.py#L25-L61","documentation":"chunk_embedding_fn converts an EmbeddableItem's dense embedding into the MySQL vector string '[v1,v2,...]'. It requires chunk.embedding and chunk.embedding.dense_embedding to be present; if either is None it raises, because a vector column cannot be written without a dense vector.","triggerScenarios":"Writing to MySQL via a ColumnSpec whose value_fn is chunk_embedding_fn when a chunk was never embedded (embedding stage failed/skipped), produced only sparse embeddings, or chunk.embedding is None.","commonSituations":"Pipeline stage ordering errors where the MySQL sink runs before the embedding transform; an embedding model returning None for empty text; loading pre-existing chunks that lack embeddings.","solutions":["Ensure an embedding transform (e.g. MLTransform or an EmbeddingFn) runs on all chunks before the MySQL writer.","Check that your embedding function produces dense_embedding, not just sparse_embedding.","Log/inspect chunks upstream and filter out (or re-embed) items with chunk.embedding is None."],"exampleFix":"// before\nrows = chunks | MySqlVectorWriter(config)  # chunks not embedded\n// after\nembedded = chunks | \"embed\" >> MLTransform(...).with_transform(embedding_transform)\nrows = embedded | MySqlVectorWriter(config)","handlingStrategy":"type-guard","validationCode":"if chunk.embedding is None or chunk.embedding.dense_embedding is None:\n    raise ValueError(f\"chunk missing dense embedding: {chunk.id}\")","typeGuard":"def has_dense_embedding(chunk) -> bool:\n    return chunk.embedding is not None and chunk.embedding.dense_embedding is not None","tryCatchPattern":"try:\n    vec = chunk_embedding_fn(chunk)\nexcept ValueError:\n    chunk = re_embed(chunk)  # or route to dead-letter\n    vec = chunk_embedding_fn(chunk)","preventionTips":["Run the embedding stage immediately before the sink in the pipeline graph","Assert dense embeddings exist in a validation DoFn before writing","Make your embedding fn raise instead of returning None on failure"],"tags":["python","mysql","rag","missing-embedding"],"backgroundTag":"null-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}