quickwit-oss/quickwit · error

failed to parse Kafka client parameters. `client_params.{}`

Error message

failed to parse Kafka client parameters. `client_params.{}` must be a boolean, number, or string

What it means

Within a `client_params` object, each value must be a JSON boolean, number, string, or null (null skips the key). Arrays and objects cannot be converted to librdkafka settings, so `parse_client_params` bails naming the offending key.

Source

Thrown at quickwit/quickwit-indexing/src/source/kafka_source.rs:721

        ),
    };
    Ok(log_level)
}

fn parse_client_params(client_params: JsonValue) -> anyhow::Result<ClientConfig> {
    let params = if let JsonValue::Object(params) = client_params {
        params
    } else {
        bail!("failed to parse Kafka client parameters. `client_params` must be a JSON object");
    };
    let mut client_config = ClientConfig::new();
    for (key, value_json) in params {
        let value = match value_json {
            JsonValue::Bool(value_bool) => value_bool.to_string(),
            JsonValue::Number(value_number) => value_number.to_string(),
            JsonValue::String(value_string) => value_string,
            JsonValue::Null => continue,
            JsonValue::Array(_) | JsonValue::Object(_) => bail!(
                "failed to parse Kafka client parameters. `client_params.{}` must be a boolean, \
                 number, or string",
                key
            ),
        };
        client_config.set(key, value);
    }
    Ok(client_config)
}

/// Returns the message payload as a `Bytes` object if it exists and is not empty.
fn message_payload_to_doc(message: &BorrowedMessage) -> Option<Bytes> {
    match message.payload() {
        Some(payload) if !payload.is_empty() => {
            let doc = Bytes::from(payload.to_vec());
            return Some(doc);
        }
        Some(_) => debug!(

View on GitHub (pinned to a39730c5cd)

Solutions

1. Flatten the offending key's value to a scalar string (librdkafka convention: comma-separated lists).
2. Remove keys with array/object values that librdkafka does not need.
3. Validate `client_params` values are only bool/number/string/null before applying the source config.

Example fix

// before
"client_params": { "bootstrap.servers": ["b1:9092", "b2:9092"] }
// after
"client_params": { "bootstrap.servers": "b1:9092,b2:9092" }

// JS
for (const [k, v] of Object.entries(params.client_params)) {
  if (!(typeof v === 'boolean' || typeof v === 'number' || typeof v === 'string' || v === null)) throw new Error(`client_params.${k} must be scalar`);
}

const isScalar = (v) => v === null || ['boolean','number','string'].includes(typeof v);

When it happens

Trigger: `create_consumer` or `check_connectivity` processes a `client_params` entry whose value is a JSON array or object, e.g. `"ssl.ca": [...]`.

Common situations: Nested config structures pasted from other tools; values like lists of brokers (`"bootstrap.servers": ["b1:9092"]`) instead of comma-separated strings.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• failed to parse Kafka client parameters. `client_params` mus
• topic `{}` does not exist
• failed to parse Kafka client log level. value `{}` is not su
• Encountered unknown command: code {other}
• unknown URI protocol `{protocol}`

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/66ed27553e479d35. Report an issue: GitHub.