pathwaycom/pathway · error · ValueError

batch_size must be a positive integer, got {batch_size}.

Error message

batch_size must be a positive integer, got {batch_size}.

What it means

pw.io.milvus.write buffers rows into batches of batch_size before upserting. A batch_size below 1 is meaningless (it would never flush), so the connector validates it eagerly at setup and raises ValueError echoing the value you passed.

Source

Thrown at python/pathway/io/milvus/__init__.py:289

    ... )

    Attach the Milvus output connector and specify which column maps to the
    Milvus primary key field:

    >>> pw.io.milvus.write(   # doctest: +SKIP
    ...     table,
    ...     uri="./milvus.db",
    ...     collection_name="docs",
    ...     primary_key=table.doc_id,
    ... )
    >>> pw.run(monitoring_level=pw.MonitoringLevel.NONE)  # doctest: +SKIP
    """
    _check_entitlements("milvusdb")
    with optional_imports("milvus"):
        from pymilvus import MilvusClient

    if batch_size < 1:
        raise ValueError(f"batch_size must be a positive integer, got {batch_size}.")

    if primary_key._table is not table:
        raise ValueError(
            f"primary_key column {primary_key._name!r} does not belong to the "
            f"provided table. Pass a column reference from the same table, "
            f"e.g. primary_key=table.{primary_key._name}."
        )

    client = _make_client(MilvusClient, uri)

    # Fail fast if the collection is missing: otherwise the error would only
    # surface deep inside pw.run() on the first upsert, or — for an empty table —
    # never, silently running a misconfigured pipeline that writes nothing.
    if not client.has_collection(collection_name):
        client.close()
        raise ValueError(
            f"Milvus collection {collection_name!r} does not exist; create it "
            f"before writing. pw.io.milvus.write never creates a collection "

View on GitHub (pinned to fa2f74a464)

Solutions

1. Set a positive integer, e.g. batch_size=100 (or simply omit it to use the default)
2. Validate configuration at startup: batch_size = max(1, int(batch_size)) or assert before building the pipeline
3. Trace where the value originates — an unset env var or integer division producing 0

Example fix

# before
pw.io.milvus.write(t, uri, collection_name="docs", primary_key=t.id, batch_size=len(rows)//10)  # 0 for small tables

# after
batch_size = max(1, len(rows)//10)
pw.io.milvus.write(t, uri, collection_name="docs", primary_key=t.id, batch_size=batch_size)

batch_size = int(batch_size)
if batch_size < 1:
    raise ValueError(f"batch_size must be >= 1, got {batch_size}")
# or normalize: batch_size = max(1, int(batch_size))

def is_valid_batch_size(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) and v >= 1

When it happens

Trigger: Passing batch_size=0 (often as an unintentional default from config), a negative number, or a value computed from another variable that evaluates to 0 or below.

Common situations: Reading batch_size from env/config where an unset value coerces to 0; arithmetic like len(rows)//N evaluating to 0 for small inputs; copy-paste of an example with a placeholder.

Related errors

• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• schema does not match given dataframe
• negative timestamp cannot be used
• demo.range_stream error: nb_rows should be strictly positive

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/db97079413351abd. Report an issue: GitHub.