{"record":{"id":"4f5d31469f6e1f65","repo":"apache/beam","slug":"item-embeddable-item-id-missing-dense-embedding-required-for","errorCode":null,"errorMessage":"Item {embeddable_item.id} missing dense embedding required for vector search","messagePattern":"Item (.+?) missing dense embedding required for vector search","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/rag/enrichment/milvus_search.py","lineNumber":489,"sourceCode":"        data=vector_search_data,\n        anns_field=self.search_strategy.vector.anns_field,\n        param=self.search_strategy.vector.search_params,\n        limit=self.search_strategy.vector.limit,\n        expr=self.search_strategy.vector.filter)\n\n    keyword_search_req = AnnSearchRequest(\n        data=keyword_search_data,\n        anns_field=self.search_strategy.keyword.anns_field,\n        param=self.search_strategy.keyword.search_params,\n        limit=self.search_strategy.keyword.limit,\n        expr=self.search_strategy.keyword.filter)\n\n    reqs = [vector_search_req, keyword_search_req]\n    return reqs\n\n  def _get_vector_search_data(self, embeddable_item: EmbeddableItem):\n    if not embeddable_item.dense_embedding:\n      raise ValueError(\n          f\"Item {embeddable_item.id} missing dense embedding required for\"\n          \" vector search\")\n    return embeddable_item.dense_embedding\n\n  def _get_keyword_search_data(self, embeddable_item: EmbeddableItem):\n    has_no_text = not embeddable_item.content.text\n    has_no_sparse = not embeddable_item.sparse_embedding\n    if has_no_text and has_no_sparse:\n      raise ValueError(\n          f\"Item {embeddable_item.id} missing both text content and sparse \"\n          \"embedding required for keyword search\")\n    sparse_embedding = MilvusHelpers.sparse_embedding(\n        embeddable_item.sparse_embedding)\n    return embeddable_item.content.text or sparse_embedding\n\n  def _get_call_response(\n      self,\n      embeddable_items: list[EmbeddableItem],","sourceCodeStart":471,"sourceCodeEnd":507,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/rag/enrichment/milvus_search.py#L471-L507","documentation":"When running vector (or hybrid) search, MilvusEnricher converts each EmbeddableItem to a query vector via _get_vector_search_data. If an item's dense_embedding is empty/None, it cannot be used as a Milvus query vector, so a ValueError names the offending item id.","triggerScenarios":"An EmbeddableItem flowing through MilvusEnricher with hybrid or vector search strategy whose dense_embedding field is None or empty — usually because the upstream embedding step did not run or failed for that item.","commonSituations":"Embedding transform skipped for some records (e.g., empty text failing silently); items loaded from a cache/source that lacks embeddings; mixing strategies where keyword-sourced items enter a vector search pipeline.","solutions":["Ensure every item passes through an embedding transform (e.g., EmbedText) before MilvusEnricher in vector/hybrid mode","Filter out or re-embed items with empty dense_embedding before enrichment","Log/inspect which ids lack embeddings; fix upstream producer for those records"],"exampleFix":"// before\nresults = pcoll | MilvusEnricher(params)  # items not embedded\n// after\ndef has_dense(item):\n    return item.embedding and item.embedding.dense_embedding\nresults = (pcoll\n    | 'FilterEmbedded' >> beam.Filter(has_dense)\n    | MilvusEnricher(params))","handlingStrategy":"validation","validationCode":"bad = [it.id for it in items if not (it.embedding and it.embedding.dense_embedding)]\nassert not bad, f'items missing dense embedding: {bad}'","typeGuard":"def has_dense_embedding(item) -> bool:\n    return bool(item.embedding and item.embedding.dense_embedding)","tryCatchPattern":"try:\n    enriched = enricher(items)\nexcept ValueError as e:\n    if 'missing dense embedding' in str(e): re_embed_and_retry(items)\n    else: raise","preventionTips":["Always embed items upstream before vector/hybrid Milvus enrichment","Add a beam.Filter for dense embeddings before the enricher","Monitor embedding-step output counts for dropped records"],"tags":["python","apache-beam","milvus","rag","embeddings"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}