apache/beam · error · ValueError

Item missing both text content and sparse embedding…

Error message

Item {embeddable_item.id} missing both text content and sparse embedding required for keyword search

What it means

For keyword (or hybrid) search, _get_keyword_search_data requires each EmbeddableItem to supply at least one query input: text content or a sparse embedding. If both are absent, Milvus has nothing to search with and the enricher raises this ValueError naming the item id.

Solutions

1. Populate item.content.text (or the sparse embedding) before enrichment
2. Add sparse embedding generation to the pipeline when using keyword/hybrid search
3. Filter out empty-content items before the enricher

Example fix

// before
item = EmbeddableItem(id='1', content=Content(text=''))  # no sparse either
// after
item = EmbeddableItem(id='1', content=Content(text='chunk body'))
# or provide item.embedding.sparse_embedding via a sparse embedder

bad = [it.id for it in items if not it.content.text and not it.sparse_embedding]
assert not bad, f'items missing text and sparse embedding: {bad}'

def keyword_searchable(item) -> bool:
    return bool(item.content.text) or bool(item.sparse_embedding)

try:
    enriched = enricher(items)
except ValueError as e:
    if 'missing both text content and sparse' in str(e): fallback_to_vector_search(items)
    else: raise

When it happens

Trigger: An EmbeddableItem with empty content.text and empty/None sparse_embedding flows through MilvusEnricher configured with KeywordSearchStrategy or HybridSearchStrategy.

Common situations: Documents with empty bodies ingested from a sparse source; sparse embedding step omitted in the pipeline; hybrid search where only dense embeddings were generated and text was cleared after chunking.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Item missing dense embedding required for vector search
• EmbeddableItem must contain dense embedding
• Not supported search strategy yet
• Search strategy must be provided
• Approximate Nearest Neighbor Search (ANNS) field must be…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/658be9cd70483fe8. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/rag/enrichment/milvus_search.py:498

        param=self.search_strategy.keyword.search_params,
        limit=self.search_strategy.keyword.limit,
        expr=self.search_strategy.keyword.filter)

    reqs = [vector_search_req, keyword_search_req]
    return reqs

  def _get_vector_search_data(self, embeddable_item: EmbeddableItem):
    if not embeddable_item.dense_embedding:
      raise ValueError(
          f"Item {embeddable_item.id} missing dense embedding required for"
          " vector search")
    return embeddable_item.dense_embedding

  def _get_keyword_search_data(self, embeddable_item: EmbeddableItem):
    has_no_text = not embeddable_item.content.text
    has_no_sparse = not embeddable_item.sparse_embedding
    if has_no_text and has_no_sparse:
      raise ValueError(
          f"Item {embeddable_item.id} missing both text content and sparse "
          "embedding required for keyword search")
    sparse_embedding = MilvusHelpers.sparse_embedding(
        embeddable_item.sparse_embedding)
    return embeddable_item.content.text or sparse_embedding

  def _get_call_response(
      self,
      embeddable_items: list[EmbeddableItem],
      search_result: SearchResult[Hits]):
    response = []
    for i in range(len(embeddable_items)):
      embeddable_item = embeddable_items[i]
      hits: Hits = search_result[i]
      result = MilvusSearchResult()
      for j in range(len(hits)):
        hit: Hit = hits[j]
        normalized_fields = self._normalize_milvus_fields(hit.fields)

View on GitHub (pinned to 12126d8942)