What causes "Collection name must be provided" in apache/beam?

Pass a non-empty collection_name when constructing the sink config

apache/beam · error · ValueError

Collection name must be provided

Error message

Collection name must be provided

What it means

The Qdrant write sink dataclass requires collection_name in __post_init__; without it Qdrant would not know which collection to upsert points into. The library fails fast at config construction instead of mid-pipeline.

Solutions

Pass a non-empty collection_name when constructing the sink config
Check that the value isn't an empty string (which also fails this check)
Create the collection in Qdrant first and use its exact name
If read from options, add a default or assert it's set before building the pipeline

Example fix

// before
params = QdrantWriteParameters(host=..., port=6333)
// after
params = QdrantWriteParameters(host=..., port=6333, collection_name="rag_chunks")

Defensive patterns

Strategy: validation

Validate before calling

if not collection_name:
    raise ValueError("Set --qdrant_collection before building the pipeline")

Type guard

def has_collection(params) -> bool:
    return isinstance(params.collection_name, str) and params.collection_name != ""

Prevention

Give collection_name a required pipeline option with no default
Verify the collection exists in Qdrant with the exact same name
Avoid empty-string defaults when plumbing options into the sink config

When it happens

Trigger: Constructing the Qdrant sink config (e.g. QdrantWriteParameters / CustomOptionsJunction) with collection_name=None or empty string '', then calling create_write_transform() or adding the sink to a Beam pipeline.

Common situations: Forgetting to pass collection_name through pipeline options; typos like collection='items'; building configs from env vars where the collection variable is unset; running against a recreated cluster where the intended collection name was lost from config.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3657b66266c18241. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/rag/ingestion/qdrant.py:199

    timeout: Optional timeout for write operations in seconds. Default is None.
    batch_size: Number of points to write in each batch. Default is 1000.
    kwargs: Additional keyword arguments to pass to the client's upsert method.
    dense_embedding_key: name for the dense vector in the qdrant collection.
    sparse_embedding_key: name for the sparse vector in the qdrant collection.
  """

  connection_params: QdrantConnectionParameters
  collection_name: str
  timeout: Optional[int] = None
  batch_size: int = DEFAULT_WRITE_BATCH_SIZE
  max_batch_byte_size: int = DEFAULT_MAX_BATCH_BYTE_SIZE
  kwargs: dict[str, Any] = field(default_factory=dict)
  dense_embedding_key: str = "dense"
  sparse_embedding_key: str = "sparse"

  def __post_init__(self):
    if not self.collection_name:
      raise ValueError("Collection name must be provided")
    if self.batch_size <= 0:
      raise ValueError("Batch size must be a positive integer")

  def create_write_transform(self) -> beam.PTransform[EmbeddableItem, Any]:
    return _QdrantWriteTransform(self)

  def create_converter(
      self,
  ) -> Callable[[EmbeddableItem], "models.PointStruct"]:
    def convert(item: EmbeddableItem) -> "models.PointStruct":
      if item.dense_embedding is None and item.sparse_embedding is None:
        raise ValueError(
            "EmbeddableItem must have at least one embedding (dense or sparse)")
      vector = {}
      if item.dense_embedding is not None:
        vector[self.dense_embedding_key] = item.dense_embedding
      if item.sparse_embedding is not None:
        sparse_indices, sparse_values = item.sparse_embedding

View on GitHub (pinned to 12126d8942)