risingwavelabs/risingwave · error · SinkError::Http

Turbopuffer sink received non-success response

Error message

Turbopuffer sink received non-success response: {} {}

What it means

send_turbopuffer_request performs the HTTP POST/DELETE to the Turbopuffer v2 API and requires a 2xx status. If the API returns any non-success status, the error includes the HTTP status code and the response body text. This surfaces auth failures, invalid payloads, quota issues, and Turbopuffer-side validation errors.

Solutions

  1. Read the status and body in the error message to identify the cause (401/403 → fix the API key; 400 → fix schema/payload; 429 → reduce throughput).
  2. Verify the API key and that the namespace/project are correct.
  3. Check that column types in the sink match the Turbopuffer namespace schema; drop or recreate the namespace if the schema changed.
  4. Retry with backoff for 429/5xx responses.

Example fix

// before: expired key
WITH (connector='turbopuffer', turbopuffer.api_key='sk-old', ...)
// after: valid key via secret
WITH (connector='turbopuffer', turbopuffer.api_key=secret turbopuffer_api_key, ...)
Defensive patterns

Strategy: retry

Validate before calling

// before creating the sink, verify credentials with a direct API call:
// curl -s -o /dev/null -w '%{http_code}' -H "Authorization: Bearer $TPUF_KEY" https://api.turbopuffer.com/v1/namespaces

Try / catch

match res {
    Ok(()) => {}
    Err(e) if e.to_string().contains("non-success response: 429") => retry_with_backoff(e),
    Err(e) => return Err(e), // 4xx: do not retry, fix key/schema
}

Prevention

When it happens

Trigger: flush_all sends a buffered write/delete payload and the Turbopuffer API responds with a non-2xx status (401 wrong API key, 400 malformed payload/schema mismatch, 429 rate limited, 5xx server error).

Common situations: Expired or wrong TURBOPUFFER_API_KEY; schema in the upsert not matching an existing namespace's schema; exceeding rate limits during heavy streaming writes; network/proxy returning error pages.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/8d082b32939fec92. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/sink/turbopuffer.rs:678

        }))
        .await?;
        Ok(())
    }
}

async fn send_turbopuffer_request(client: reqwest::Client, url: String, body: Value) -> Result<()> {
    let resp = client
        .post(url)
        .json(&body)
        .send()
        .await
        .context("turbopuffer write request failed")
        .map_err(SinkError::Http)?;

    if !resp.status().is_success() {
        let status = resp.status();
        let body = resp.text().await.unwrap_or_default();
        return Err(SinkError::Http(anyhow!(
            "Turbopuffer sink received non-success response: {} {}",
            status,
            body
        )));
    }
    Ok(())
}

#[derive(Clone, Debug, Eq, Hash, PartialEq, Serialize)]
#[serde(untagged)]
enum DocumentId {
    U64(u64),
    String(String),
}

#[derive(Debug)]
enum CompactedOp {
    Upsert(Map<String, Value>),

View on GitHub (pinned to 6469eb736d)