apache/beam · error · ValueError

EmbeddableItem must have at least one embedding (dense or…

Error message

EmbeddableItem must have at least one embedding (dense or sparse)

What it means

The converter built by create_converter() maps each EmbeddableItem to a Qdrant PointStruct. If an item has neither dense_embedding nor sparse_embedding, there is no vector to store, so the per-item conversion raises ValueError.

Solutions

Run the items through an embedding generation transform (e.g. the embedding config in the RAG pipeline) before the Qdrant sink
Filter out un-embedded items before writing
Check the embedding function/model for cases that return None (e.g. empty text)
Populate at least dense_embedding, or sparse_embedding as (indices, values) for sparse models

Example fix

// before
items | beam.Map(load_chunk) | qdrant_sink
// after
items | beam.Map(load_chunk) | embed_transform | qdrant_sink

Defensive patterns

Strategy: try-catch

Validate before calling

def is_embeddable(item) -> bool:
    return item.dense_embedding is not None or item.sparse_embedding is not None
items = [i for i in items if is_embeddable(i)]

Type guard

def has_vector(item) -> bool:
    return item.dense_embedding is not None or item.sparse_embedding is not None

Try / catch

try:
    point = converter(item)
except ValueError as e:
    logging.warning("Skipping item without embedding: %s", item)
    return None

Prevention

Always place the embedding-generation transform before the Qdrant sink
Filter un-embedded items before writing
Check embedding functions for inputs that yield None (empty strings, blank pages)

When it happens

Trigger: Feeding items into the Qdrant write transform where the embedding step was skipped, the embedding model returned None, or items come from a source that doesn't populate embeddings (e.g. test fixtures or filtered records).

Common situations: Upstream EmbeddingsGeneration PTransform removed or failing silently; embedding API returning nulls for empty inputs; mixing pre-chunked items with items not yet embedded in one stream.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a0ee36660040bf84. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/rag/ingestion/qdrant.py:211

  kwargs: dict[str, Any] = field(default_factory=dict)
  dense_embedding_key: str = "dense"
  sparse_embedding_key: str = "sparse"

  def __post_init__(self):
    if not self.collection_name:
      raise ValueError("Collection name must be provided")
    if self.batch_size <= 0:
      raise ValueError("Batch size must be a positive integer")

  def create_write_transform(self) -> beam.PTransform[EmbeddableItem, Any]:
    return _QdrantWriteTransform(self)

  def create_converter(
      self,
  ) -> Callable[[EmbeddableItem], "models.PointStruct"]:
    def convert(item: EmbeddableItem) -> "models.PointStruct":
      if item.dense_embedding is None and item.sparse_embedding is None:
        raise ValueError(
            "EmbeddableItem must have at least one embedding (dense or sparse)")
      vector = {}
      if item.dense_embedding is not None:
        vector[self.dense_embedding_key] = item.dense_embedding
      if item.sparse_embedding is not None:
        sparse_indices, sparse_values = item.sparse_embedding
        vector[self.sparse_embedding_key] = models.SparseVector(
            indices=sparse_indices,
            values=sparse_values,
        )
      id = (
          int(item.id)
          if isinstance(item.id, str) and item.id.isdigit() else item.id)
      return models.PointStruct(
          id=id,
          vector=vector,
          payload=item.metadata if item.metadata else None,
      )

View on GitHub (pinned to 12126d8942)