pathwaycom/pathway · error · ValueError

batch_size must be a positive integer, got {batch_size}.

Error message

batch_size must be a positive integer, got {batch_size}.

What it means

Raised by pw.io.pinecone.write() when batch_size is zero or negative. The value is forwarded to the engine as max_batch_size for the Pinecone upsert batches, which must contain at least one record.

Source

Thrown at python/pathway/io/pinecone/__init__.py:385

    else:
        metadata_names = []
        for col in metadata_columns:
            if col._table is not table:
                raise ValueError(
                    f"metadata column {col._name!r} does not belong to the "
                    f"provided table. Pass column references from the same table, "
                    f"e.g. table.{col._name}."
                )
            if col._name in (pk_name, vector_name):
                role = "primary_key" if col._name == pk_name else "vector"
                raise ValueError(
                    f"column {col._name!r} is used as the {role} and cannot also "
                    "be a metadata column. Remove it from metadata_columns."
                )
            metadata_names.append(col._name)

    if batch_size <= 0:
        raise ValueError(f"batch_size must be a positive integer, got {batch_size}.")

    if pk_name is not None:
        _check_primary_key_dtype(pk_name, table._get_column(pk_name).dtype)
    _check_vector_dtype(vector_name, table._get_column(vector_name).dtype)
    for col_name in metadata_names:
        _check_metadata_dtype(col_name, table._get_column(col_name).dtype)

    resolved_api_key = (
        api_key if api_key is not None else os.environ.get("PINECONE_API_KEY")
    )
    if not resolved_api_key:
        raise ValueError(
            "A Pinecone API key is required. Pass api_key=... or set the "
            "PINECONE_API_KEY environment variable."
        )

    column_index = {name_: index for index, name_ in enumerate(table.column_names())}
    vector_index = column_index[vector_name]

View on GitHub (pinned to fa2f74a464)

Solutions

1. Set a positive batch_size (typical values 50–200 for Pinecone upserts).
2. If batch_size is computed, clamp it: batch_size = max(1, computed).
3. Omit batch_size to use the connector default.

Example fix

# before
pw.io.pinecone.write(t, "docs", vector=t.emb, api_key=k, batch_size=0)
# after
pw.io.pinecone.write(t, "docs", vector=t.emb, api_key=k, batch_size=100)

batch_size = max(1, int(batch_size))
assert batch_size > 0

When it happens

Trigger: Passing batch_size=0, a negative number, or a computed value (e.g. len(some_list) that evaluates to 0, or a config-tuned value that fell to 0) to pw.io.pinecone.write().

Common situations: batch_size derived from environment variables or an adaptive formula; typos like batch_size=-1 meaning 'unbounded' (not supported here).

Related errors

• Batching is not supported for fully asynchronous UDFs.
• A batch UDF has to return a list but is annotated as returni
• primary_key column {name!r} is nullable (type {dtype}); a Pi
• primary_key column {name!r} has unsupported type {dtype}; a
• vector column {name!r} is nullable (type {dtype}); every row

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/7782cbca2a7833cb. Report an issue: GitHub.