pathwaycom/pathway · error · ValueError

rdkafka_settings must contain a non-empty 'bootstrap.servers

Error message

rdkafka_settings must contain a non-empty 'bootstrap.servers' entry so the producer can locate a broker; got {rdkafka_settings.get('bootstrap.servers')!r}.

What it means

pw.io.kafka.write validates that the producer's rdkafka_settings contains a non-empty 'bootstrap.servers' entry — without a broker address the librdkafka producer cannot deliver messages. The error echoes the offending value so you can see what was actually found (usually None or '').

Source

Thrown at python/pathway/io/kafka/__init__.py:711

    ...     value=t2.foo,
    ... )

    Still, the table has three fields and the field ``baz`` is not produced. You can do it
    with the usage of headers. To pass it to the header with the same name ``baz``, you need to
    specify it:

    >>> pw.io.kafka.write(
    ...     t2,
    ...     rdkafka_settings,
    ...     "test",
    ...     format="raw",
    ...     key=t2.bar,
    ...     value=t2.foo,
    ...     headers=[t2.baz],
    ... )
    """
    if not rdkafka_settings.get("bootstrap.servers"):
        raise ValueError(
            "rdkafka_settings must contain a non-empty 'bootstrap.servers' "
            "entry so the producer can locate a broker; got "
            f"{rdkafka_settings.get('bootstrap.servers')!r}."
        )
    if isinstance(topic_name, str) and not topic_name:
        raise ValueError(
            "'topic_name' must be a non-empty string; got an empty string. "
            "Kafka does not allow empty topic names."
        )

    output_format = MessageQueueOutputFormat.construct(
        table,
        format=format,
        delimiter=delimiter,
        key=key,
        value=value,
        headers=headers,
        topic_name=topic_name if isinstance(topic_name, ColumnReference) else None,

View on GitHub (pinned to fa2f74a464)

Solutions

1. Add the broker list: rdkafka_settings = {'bootstrap.servers': 'localhost:9092'} (no group.id needed for producers).
2. Verify the key is exactly 'bootstrap.servers'.
3. Fail fast at startup when the address is missing rather than discovering it at first write.

Example fix

# before
pw.io.kafka.write(t, {}, "test", format="raw")
# after
pw.io.kafka.write(
    t,
    {"bootstrap.servers": "localhost:9092"},
    "test",
    format="raw",
)

def require_producer_settings(settings: dict) -> None:
    if not settings.get("bootstrap.servers"):
        raise SystemExit("producer settings need a non-empty 'bootstrap.servers'")

require_producer_settings(rdkafka_settings)
pw.io.kafka.write(table, rdkafka_settings, "test", format="raw")

def producer_settings_ok(settings: dict) -> bool:
    return bool(settings.get("bootstrap.servers"))

When it happens

Trigger: pw.io.kafka.write(table, {}, 'test') — settings dict missing 'bootstrap.servers'; or {'bootstrap.servers': ''}; or the key spelled 'bootstrap_servers'/'bootstrap.server'. This is the write-side twin of the reader's check.

Common situations: Reusing reader settings but stripping the broker entry; loading settings from environment/config where the address variable was empty; typo in the dotted librdkafka key name.

Related errors

• rdkafka_settings must contain a non-empty 'bootstrap.servers
• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• 'subject' must be a non-empty string; got an empty string. S
• rdkafka_settings must contain a non-empty 'group.id' entry:

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/3ab1486d0c93f61b. Report an issue: GitHub.