{"record":{"id":"efa5656cdd8154d7","repo":"apache/beam","slug":"embeddableitem-must-contain-embedding-embeddable","errorCode":null,"errorMessage":"EmbeddableItem must contain embedding: {embeddable}","messagePattern":"EmbeddableItem must contain embedding: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/rag/ingestion/spanner.py","lineNumber":224,"sourceCode":"    \n    Examples:\n        Default embedding:\n        >>> builder.with_embedding_spec()\n        \n        Normalized embedding:\n        >>> def normalize(vec):\n        ...     norm = (sum(x**2 for x in vec) ** 0.5) or 1.0\n        ...     return [x/norm for x in vec]\n        >>> builder.with_embedding_spec(convert_fn=normalize)\n        \n        Rounded precision:\n        >>> builder.with_embedding_spec(\n        ...     convert_fn=lambda vec: [round(x, 4) for x in vec]\n        ... )\n    \"\"\"\n    def extract_fn(embeddable: EmbeddableItem) -> list[float]:\n      if not embeddable.dense_embedding:\n        raise ValueError(f'EmbeddableItem must contain embedding: {embeddable}')\n      return embeddable.dense_embedding\n\n    self._specs.append(\n        SpannerColumnSpec(\n            column_name=column_name,\n            python_type=list[float],\n            value_fn=functools.partial(\n                _extract_and_convert, extract_fn, convert_fn)))\n    return self\n\n  def with_content_spec(\n      self,\n      column_name: str = \"content\",\n      python_type: type = str,\n      convert_fn: Optional[Callable[[str], Any]] = None\n  ) -> 'SpannerColumnSpecsBuilder':\n    \"\"\"Add content column.\n    ","sourceCodeStart":206,"sourceCodeEnd":242,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/rag/ingestion/spanner.py#L206-L242","documentation":"The default extract_fn used by SpannerColumnSpec.with_embedding_spec requires the EmbeddableItem to have a dense_embedding; if it's empty/None there is no vector to convert for the Spanner column, so it raises with the offending item in the message.","triggerScenarios":"Writing to Cloud Spanner via the SpannerVectorWriterBuilder without running an embedding transform first, or with items whose dense_embedding is None or an empty list (empty lists are falsy and also trigger this).","commonSituations":"Embedding model returning [] for empty text; records sourced before the embedding stage; supplying a custom convert_fn but relying on the default extract_fn for items that only carry sparse embeddings.","solutions":["Ensure an embedding-generation transform populates dense_embedding before the Spanner sink","Supply a custom extract_fn to with_embedding_spec that reads a different field or raises later","Filter out items with empty dense_embedding before writing","Debug why the embedding step produced empty vectors for the offending item (logged in the message)"],"exampleFix":"// before\nbuilder.with_embedding_spec(column_name='embedding')\n// after (custom extraction)\nbuilder.with_embedding_spec(\n    column_name='embedding',\n    extract_fn=lambda item: item.dense_embedding or fallback_embed(item.content.text))","handlingStrategy":"validation","validationCode":"bad = [i for i in items if not i.dense_embedding]\nif bad:\n    raise ValueError(f\"{len(bad)} items lack dense_embedding before Spanner write\")","typeGuard":"def has_dense(item) -> bool:\n    return bool(item.dense_embedding)","tryCatchPattern":"try:\n    row = spec.to_row(item)\nexcept ValueError:\n    logging.warning(\"Skipping item without embedding: %s\", item)\n    return None","preventionTips":["Run the embedding transform before the Spanner sink in the pipeline graph","Filter items with empty dense_embedding lists (empty lists also fail)","Log un-embedded items upstream so gaps are visible before the write stage"],"tags":["python","spanner","embeddings","data-validation"],"backgroundTag":"empty-required-field","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}