pathwaycom/pathway · error · ValueError

batch_size must be a positive integer, got {batch_size}.

Error message

batch_size must be a positive integer, got {batch_size}.

What it means

Raised by pw.io.qdrant.write when batch_size is less than 1. The value is forwarded to the Qdrant params and controls how many points are upserted per request; zero or negative batches are meaningless and would break the upsert loop or the server request, so it is validated up front.

Source

Thrown at python/pathway/io/qdrant/__init__.py:138

    ...         (1, [0.1, 0.2, 0.3, 0.4], [(7, 2.0), (21, 1.0)], "a"),
    ...         (2, [0.5, 0.6, 0.7, 0.8], [(3, 1.0)], "b"),
    ...     ],
    ... )
    >>> pw.io.qdrant.write(   # doctest: +SKIP
    ...     table,
    ...     url="http://localhost:6334",
    ...     collection_name="docs",
    ... )
    >>> pw.run(monitoring_level=pw.MonitoringLevel.NONE)  # doctest: +SKIP

    If the collection declares only the dense ``embedding`` slot, drop the
    ``bm25`` column from the schema above — this is the common non-hybrid
    setup.
    """
    _check_entitlements("qdrant")

    if batch_size < 1:
        raise ValueError(f"batch_size must be a positive integer, got {batch_size}.")

    data_storage = api.DataStorage(
        storage_type="qdrant",
        qdrant_params=api.QdrantParams(
            url=url,
            collection_name=collection_name,
            api_key=api_key,
            batch_size=batch_size,
        ),
    )
    data_format = api.DataFormat(
        format_type="identity",
        key_field_names=[],
        value_fields=_format_output_value_fields(table),
    )

    table.to(
        datasink.GenericDataSink(

View on GitHub (pinned to fa2f74a464)

Solutions

1. Set batch_size to a positive integer (e.g. the default or something like 64-512 depending on payload size).
2. Validate config at load time: parse the value and assert it is >= 1 before wiring the sink.
3. Treat 0 as 'use default' explicitly if your config convention uses it: batch_size = cfg.batch or DEFAULT_BATCH.

Example fix

# before
pw.io.qdrant.write(t, url="http://localhost:6334", collection_name="docs", batch_size=0)

# after
pw.io.qdrant.write(t, url="http://localhost:6334", collection_name="docs", batch_size=128)

batch_size = int(os.environ.get("QDRANT_BATCH", 128))
assert batch_size >= 1, f"batch_size must be >= 1, got {batch_size}"
pw.io.qdrant.write(t, url=url, collection_name=col, batch_size=batch_size)

def valid_batch_size(n) -> bool:
    return isinstance(n, int) and n >= 1

When it happens

Trigger: Calling pw.io.qdrant.write(t, url=..., collection_name=..., batch_size=0) or a negative value, typically from a config/env variable or a computed default that underflows.

Common situations: batch_size read from an environment variable that defaults to 0 when unset; tuning scripts sweeping batch sizes starting at 0; config files shared across tools where 0 means 'auto' elsewhere but is invalid here.

Related errors

• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• schema does not match given dataframe
• negative timestamp cannot be used
• demo.range_stream error: nb_rows should be strictly positive

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/094f2ea7d08e7f14. Report an issue: GitHub.