risingwavelabs/risingwave · error

The number of broker addrs {} does not match the number of p

Error message

The number of broker addrs {} does not match the number of private link targets {}

What it means

When Kafka brokers are accessed through AWS PrivateLink, each broker address must map 1:1 to a private link target (ENI). This error is raised when the number of broker addresses in `broker_addrs` differs from the number of entries parsed from the `privatelink.targets` JSON, making the rewrite map ambiguous.

Source

Thrown at src/connector/src/source/kafka/private_link.rs:135

        .map_err(Into::into)
}

pub fn insert_privatelink_broker_rewrite_map(
    with_options: &mut BTreeMap<String, String>,
    svc: Option<&PrivateLinkService>,
    privatelink_endpoint: Option<String>,
) -> ConnectorResult<()> {
    let mut broker_rewrite_map = HashMap::new();
    let servers = get_property_required(with_options, kafka_props_broker_key(with_options))?;
    let broker_addrs = servers.split(',').collect_vec();
    let link_target_value = get_property_required(with_options, PRIVATE_LINK_TARGETS_KEY)?;
    let link_targets: Vec<AwsPrivateLinkItem> =
        serde_json::from_str(link_target_value.as_str()).map_err(|e| anyhow!(e))?;
    // remove the private link targets from WITH options, as they are useless after we constructed the rewrite mapping
    with_options.remove(PRIVATE_LINK_TARGETS_KEY);

    if broker_addrs.len() != link_targets.len() {
        bail!(
            "The number of broker addrs {} does not match the number of private link targets {}",
            broker_addrs.len(),
            link_targets.len()
        );
    }

    if let Some(endpoint) = privatelink_endpoint {
        // new syntax: endpoint can either be a string or a json array of strings
        // if it is a string, rewrite all broker addresses to the same endpoint
        // eg. privatelink.endpoint='some_url' ==> broker1:9092 -> some_url:9092, broker2:9093 -> some_url:9093
        // if it is a json array, rewrite each broker address to the corresponding endpoint
        // eg. privatelink.endpoint = '[{"host": "aaaa"}, {"host": "bbbb"}, {"host": "cccc"}]'
        // ==> broker1:9092 -> aaaa:9092, broker2:9093 -> bbbb:9093, broker3:9094 -> cccc:9094
        handle_privatelink_endpoint(
            &endpoint,
            &mut broker_rewrite_map,
            &link_targets,
            &broker_addrs,

View on GitHub (pinned to 6469eb736d)

Solutions

1. Update the `privatelink.targets` JSON so it has exactly one entry per broker address
2. Ensure every broker address (all bootstrap broker host:ports) is included in both lists
3. Regenerate targets from the current MSK cluster info if brokers were added
4. Remove stale targets left over from a previous cluster configuration

Example fix

// before (2 targets for 3 brokers)
privatelink.targets = '[{"host":"vpce-a"},{"host":"vpce-b"}]'
// after
privatelink.targets = '[{"host":"vpce-a"},{"host":"vpce-b"},{"host":"vpce-c"}]'

let targets: Vec<_> = serde_json::from_str::<Vec<serde_json::Value>>(&targets_json)?;
if targets.len() != broker_addrs.len() {
    return Err(format!("need {} privatelink targets, got {}", broker_addrs.len(), targets.len()));
}

When it happens

Trigger: `insert_privatelink_broker_rewrite_map` (via `resolve_privatelink_in_with_option`) is called and `broker_addrs.len() != link_targets.len()` — e.g. user provided 3 brokers but 2 private link targets in the `privatelink.targets` JSON.

Common situations: MSK cluster scaled to more brokers after targets were configured; targets JSON truncated when pasting; user listed only one target but gave multiple bootstrap broker addresses; stale targets after cluster re-creation.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• PrivateLink endpoint not found
• expected JSON in the form {{"host": "endpoint url"}}, but go
• missing aws region
• No available private link endpoints for Kafka broker {}
• Both `access_key` and `secret_key` must be provided

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/2b1e86cbfb1c0a9e. Report an issue: GitHub.