risingwavelabs/risingwave · error · SinkError

sending stream load request failed

Error message

sending stream load request failed

What it means

This error wraps a reqwest client failure while executing the Doris/StarRocks stream load HTTP request inside send_stream_load_request. It fires when the HTTP request itself could not be completed at the transport level (connection failed, DNS resolution failed, timeout, TLS error), before any HTTP response is available. The original reqwest error is preserved as the anyhow context.

Source

Thrown at src/connector/src/sink/doris_starrocks_connector.rs:421

/// Please note, the FE address that user specified might be a FE follower not the leader, in this case,
/// the follower FE will redirect request to leader FE and then to BE.
async fn send_stream_load_request(
    client: Client,
    mut request: Request,
    fe_host: &str,
) -> Result<StreamLoadResponse> {
    // possible redirection paths:
    // RW <-> follower FE -> leader FE -> BE
    // RW <-> leader FE -> BE
    // RW <-> leader FE
    for _ in 0..2 {
        let original_http_port = request.url().port();
        let mut request_for_redirection = request
            .try_clone()
            .ok_or_else(|| SinkError::DorisStarrocksConnect(anyhow!("Can't clone request")))?;
        let resp = client.execute(request).await.map_err(|err| {
            SinkError::DorisStarrocksConnect(
                anyhow!(err).context("sending stream load request failed"),
            )
        })?;
        let be_url = try_get_be_url(&resp, fe_host)?;
        match be_url {
            Some(be_url) => {
                // we used an unconventional method to detect if we are currently redirecting to FE leader, i.e.,
                // by comparing the port of the redirected url with that of the original request, if they are same, we consider
                // this is a FE address. Because in practice, no one would deploy their `StarRocks` cluster with the same
                // http port for both FE and BE. However, this is a potentially problematic assumption,
                // we may investigate a better way to do this. For example, we could use the `show backends` command to check
                // if the host of the redirected url is in the list. However, `show backends` requires
                // the system-level privilege, which could break the backward compatibility.
                let redirected_port = be_url.port();
                *request_for_redirection.url_mut() = be_url;
                if redirected_port == original_http_port {
                    // redirected to FE, continue another round.
                    request = request_for_redirection;
                } else {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Verify the connector URL uses the FE HTTP port (typically 8030 for Doris/StarRocks), not the MySQL port 9030 or BE port
2. Check network reachability from the RisingWave node: curl -v http://<fe_host>:<http_port>/api/transaction/begin
3. Confirm FE and BE processes are running (show frontends / show backends via MySQL client) and not restarting under load
4. Increase stream_load_http_timeout_ms if errors are timeouts on large loads
5. Inspect the wrapped reqwest error message (source of the anyhow chain) to distinguish connect-refused vs DNS vs timeout and fix accordingly

Example fix

// before
url = 'http://fe-host:9030'  // MySQL port, no HTTP service
// after
url = 'http://fe-host:8030'  // FE http_port

let url: reqwest::Url = endpoint.parse()?;
assert!(matches!(url.scheme(), "http" | "https"), "endpoint must be http(s)");
assert!(url.host_str().is_some(), "endpoint must contain a host");
assert_ne!(url.port_or_known_default(), Some(9030), "9030 is the MySQL port; use the FE http_port (e.g. 8030)");
// then: before creating the sink, probe once:
// reqwest::get(format!("{}api/health", endpoint_trimmed)).await?;

fn is_reachable_transport_error(err: &SinkError) -> bool {
    let msg = err.to_string();
    msg.contains("sending stream load request failed")
}

match sender.send().await {
    Ok(bytes) => { /* parse stream load response */ }
    Err(e) if e.to_string().contains("sending stream load request failed") => {
        // inspect reqwest source: connect vs timeout vs dns; retry with backoff after verifying FE/BE health
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: client.execute(request) returns Err in send_stream_load_request — e.g. the FE/BE host is unreachable, DNS fails, the connection is refused or reset, TLS handshake fails, or the per-request timeout (stream_load_http_timeout for non-load requests, tokio::timeout for load requests) elapses.

Common situations: Wrong FE host/port in the sink connector URL (e.g. using the MySQL query port 9030 instead of the FE HTTP port 8030); FE/BE nodes down or restarting; network/firewall blocking the HTTP port between RisingWave and the cluster; k8s service DNS misconfiguration; long stream loads exceeding stream_load_http_timeout_ms.

Related errors

• redirection occur more than twice when sending stream load r
• Doris/Starrocks connect error: {0}
• Can't find data
• Insert error: {:?}, error url: {:?}
• Can't get doris BE url in header

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/e0b721d0684784a2. Report an issue: GitHub.