apache/beam · error · ValueError

Collection name must be provided

Error message

Collection name must be provided

What it means

MilvusSearchWriteOptions.__post_init__ validates that collection_name was set on the dataclass. The Milvus vector sink needs a target collection to write embeddings to; without it no write destination exists, so construction fails immediately.

Solutions

1. Pass collection_name='<your Milvus collection>' to MilvusSearchWriteOptions.
2. Load the collection name from config/environment and assert it is non-empty before constructing the options.
3. Verify you are not using a keyword like collection; the dataclass field is collection_name.

Example fix

// before
write_options = MilvusSearchWriteOptions(connection_config=cfg)
// after
write_options = MilvusSearchWriteOptions(
    connection_config=cfg,
    collection_name="my_rag_collection")

assert options.collection_name, "collection_name must be set before writing to Milvus"

def has_collection(o) -> bool:
    return bool(getattr(o, "collection_name", None))

try:
    options = MilvusSearchWriteOptions(**opts)
except ValueError as e:
    if "Collection name must be provided" in str(e):
        opts["collection_name"] = default_collection
        options = MilvusSearchWriteOptions(**opts)
    else:
        raise

When it happens

Trigger: Constructing MilvusSearchWriteOptions() (or passing MilvusSearchWriteOptions(connection_config=..., ...)) while omitting collection_name, or explicitly passing collection_name=None/empty string.

Common situations: Copy-pasting a write-options snippet and forgetting to fill in the collection name; building options dynamically from config where the collection key is missing; renaming a collection and leaving the variable empty.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Approximate Nearest Neighbor Search (ANNS) field must be…
• Collection name must be provided
• Item missing both text content and sparse embedding…
• Item missing dense embedding required for vector search
• Not supported search strategy yet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b2aac15bc427293f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/rag/ingestion/milvus_search.py:70

      Must be a non-empty string.
    partition_name: Name of the specific partition within the collection to
      write to. If empty, writes to the default partition.
    timeout: Maximum time in seconds to wait for write operations to complete.
      If None, uses the client's default timeout.
    write_config: Configuration for write operations including batch size and
      other write-specific settings.
    kwargs: Additional keyword arguments for write operations. Enables forward
      compatibility with future Milvus client parameters.
  """
  collection_name: str
  partition_name: str = ""
  timeout: Optional[float] = None
  write_config: WriteConfig = field(default_factory=WriteConfig)
  kwargs: dict[str, Any] = field(default_factory=dict)

  def __post_init__(self):
    if not self.collection_name:
      raise ValueError("Collection name must be provided")

  @property
  def write_batch_size(self):
    """Returns the batch size for write operations.

    Returns:
      The configured batch size, or DEFAULT_WRITE_BATCH_SIZE if not specified.
    """
    return self.write_config.write_batch_size or DEFAULT_WRITE_BATCH_SIZE


@dataclass
class MilvusVectorWriterConfig(VectorDatabaseWriteConfig):
  """Configuration for writing vector data to Milvus collections.

  This class extends VectorDatabaseWriteConfig to provide Milvus-specific
  configuration for ingesting vector embeddings and associated metadata.
  It defines how EmbeddableItem objects are converted to Milvus records and

View on GitHub (pinned to 12126d8942)