pathwaycom/pathway · error · ValueError

'parallel_readers' must be positive; got {parallel_readers}.

Error message

'parallel_readers' must be positive; got {parallel_readers}.

What it means

parallel_readers in pw.io.kafka.read controls how many parallel consumer instances share the topic's partitions; each reader needs at least one instance to make progress. Pathway raises ValueError for any explicit value <= 0 because zero or negative readers would consume nothing.

Source

Thrown at python/pathway/io/kafka/__init__.py:271

            "entry so the consumer can locate a broker; got "
            f"{rdkafka_settings.get('bootstrap.servers')!r}."
        )
    if not rdkafka_settings.get("group.id"):
        raise ValueError(
            "rdkafka_settings must contain a non-empty 'group.id' entry: "
            "Pathway's Kafka reader uses 'subscribe' (not 'assign'), which "
            "librdkafka refuses to perform without a configured consumer "
            f"group id; got {rdkafka_settings.get('group.id')!r}."
        )

    if max_backlog_size is not None and max_backlog_size <= 0:
        raise ValueError(
            f"'max_backlog_size' must be positive; got {max_backlog_size}. "
            f"A non-positive value would prevent any entry from being "
            f"processed and the reader would never make progress."
        )
    if parallel_readers is not None and parallel_readers <= 0:
        raise ValueError(
            f"'parallel_readers' must be positive; got {parallel_readers}."
        )
    if start_from_timestamp_ms is not None and start_from_timestamp_ms < 0:
        raise ValueError(
            f"'start_from_timestamp_ms' must be non-negative; got "
            f"{start_from_timestamp_ms}. The value is a Unix timestamp in "
            f"milliseconds — negative values are pre-epoch and not "
            f"meaningful for Kafka."
        )
    if autocommit_duration_ms is not None and autocommit_duration_ms <= 0:
        raise ValueError(
            f"'autocommit_duration_ms' must be positive; got "
            f"{autocommit_duration_ms}. It is the maximum time between "
            f"two commits and zero/negative values would prevent commits "
            f"from happening."
        )

    # When 'start_from_timestamp_ms' is set, the engine seeks lazily after

View on GitHub (pinned to fa2f74a464)

Solutions

1. Use parallel_readers=1 for serial reading, or match your partition count (e.g. parallel_readers=3) for parallelism.
2. Omit the parameter entirely if one reader is enough.
3. If the value is computed (e.g. min(partitions, workers)), guard it with max(1, value).

Example fix

# before
t = pw.io.kafka.read(rdkafka_settings, topic="t", parallel_readers=0)
# after
t = pw.io.kafka.read(rdkafka_settings, topic="t", parallel_readers=1)

parallel_readers = max(1, int(parallel_readers)) if parallel_readers is not None else None
t = pw.io.kafka.read(rdkafka_settings, topic="t", parallel_readers=parallel_readers)

def valid_parallel_readers(v) -> bool:
    return v is None or (isinstance(v, int) and v > 0)

When it happens

Trigger: pw.io.kafka.read(..., parallel_readers=0) or parallel_readers=-2. None (the default, meaning a single reader) is fine — only explicit non-positive integers raise.

Common situations: Setting parallel_readers=0 expecting it to mean 'automatic/serial'; deriving the count from an env var or partition count calculation that yields 0 on small topics.

Related errors

• Failed to install dependencies
• Column {pseudocolumn} has to contain integers only.
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• Unsupported argument for {data_format!r} format: 'value'
• 'delimiter' is only meaningful for the 'dsv' format, but {fo

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/fa9a7c3bcb0bffc5. Report an issue: GitHub.