pathwaycom/pathway · error · ValueError

rdkafka_settings must contain a non-empty 'bootstrap.servers

Error message

rdkafka_settings must contain a non-empty 'bootstrap.servers' entry so the consumer can locate a broker; got {rdkafka_settings.get('bootstrap.servers')!r}.

What it means

pw.io.kafka.read validates that the rdkafka_settings dict contains a non-empty 'bootstrap.servers' entry — without a broker address the librdkafka consumer cannot connect to anything. The error message includes the offending value (e.g. None or '') so you can see exactly what was found.

Source

Thrown at python/pathway/io/kafka/__init__.py:251

    >>> t = pw.io.kafka.read(
    ...     rdkafka_settings,
    ...     topic="animals",
    ...     format="json",
    ...     schema=InputSchema,
    ...     json_field_paths={
    ...         "pet_id": "/pet/identification/id",
    ...         "pet_name": "/pet/name",
    ...         "pet_height": "/pet/measurements/1"
    ...     },
    ... )

    Note that you would not need to provide the JSONPath for ``pet_id`` if it is
    at the top level of the key JSON.
    """
    # The data_storage is common to all kafka connectors

    if not rdkafka_settings.get("bootstrap.servers"):
        raise ValueError(
            "rdkafka_settings must contain a non-empty 'bootstrap.servers' "
            "entry so the consumer can locate a broker; got "
            f"{rdkafka_settings.get('bootstrap.servers')!r}."
        )
    if not rdkafka_settings.get("group.id"):
        raise ValueError(
            "rdkafka_settings must contain a non-empty 'group.id' entry: "
            "Pathway's Kafka reader uses 'subscribe' (not 'assign'), which "
            "librdkafka refuses to perform without a configured consumer "
            f"group id; got {rdkafka_settings.get('group.id')!r}."
        )

    if max_backlog_size is not None and max_backlog_size <= 0:
        raise ValueError(
            f"'max_backlog_size' must be positive; got {max_backlog_size}. "
            f"A non-positive value would prevent any entry from being "
            f"processed and the reader would never make progress."
        )

View on GitHub (pinned to fa2f74a464)

Solutions

1. Add the broker list: rdkafka_settings = {'bootstrap.servers': 'localhost:9092', 'group.id': 'my-group'} (multiple brokers comma-separated).
2. Check for typos in the key — it must be exactly 'bootstrap.servers' with a dot.
3. If settings come from config, log/fail fast when the address is empty instead of passing it through.

Example fix

# before
rdkafka_settings = {"group.id": "consumer-group"}
t = pw.io.kafka.read(rdkafka_settings, topic="t")
# after
rdkafka_settings = {
    "bootstrap.servers": "localhost:9092",
    "group.id": "consumer-group",
}
t = pw.io.kafka.read(rdkafka_settings, topic="t")

def require_bootstrap_servers(rdkafka_settings: dict) -> None:
    if not rdkafka_settings.get("bootstrap.servers"):
        raise SystemExit(
            "rdkafka_settings['bootstrap.servers'] is missing or empty"
        )

require_bootstrap_servers(rdkafka_settings)
t = pw.io.kafka.read(rdkafka_settings, topic="t")

def has_bootstrap_servers(settings: dict) -> bool:
    return bool(settings.get("bootstrap.servers"))

When it happens

Trigger: pw.io.kafka.read(rdkafka_settings={'group.id': 'g'}, topic='t') — 'bootstrap.servers' key missing; or rdkafka_settings={'bootstrap.servers': ''} with an empty string; or a typo'd key like 'bootstrap.server' or 'bootstrap_servers'.

Common situations: Loading rdkafka_settings from a config file/env where the broker address variable was not set; using the key name from a different Kafka client library (underscore vs dot); passing settings intended for pw.io.kafka.write with the address stripped out.

Related errors

• rdkafka_settings must contain a non-empty 'bootstrap.servers
• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• 'subject' must be a non-empty string; got an empty string. S
• rdkafka_settings must contain a non-empty 'group.id' entry:

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/2781c761478c8306. Report an issue: GitHub.