risingwavelabs/risingwave · warning

invalid UTF-8 in `{key}` header

Error message

invalid UTF-8 in `{key}` header

What it means

HTTP header values are not guaranteed to be UTF-8 (they are arbitrary bytes per the HTTP spec). When a webhook connector header cannot be converted with HeaderValue::to_str(), RisingWave wraps the conversion error with this context and returns 400 BAD_REQUEST. It guarantees all header-derived configuration strings are valid Rust strings.

Source

Thrown at src/frontend/src/webhook/payload.rs:191

fn parse_bool_header(headers: &HeaderMap, key: &'static str) -> Result<Option<bool>> {
    match header_value(headers, key)?.as_deref() {
        Some("true") => Ok(Some(true)),
        Some("false") => Ok(Some(false)),
        Some(value) => Err(err(
            anyhow!("unrecognized value `{value}` for `{key}`"),
            StatusCode::BAD_REQUEST,
        )),
        None => Ok(None),
    }
}

fn header_value(headers: &HeaderMap, key: &'static str) -> Result<Option<String>> {
    headers
        .get(key)
        .map(|value| {
            value.to_str().map(|value| value.to_owned()).map_err(|e| {
                err(
                    anyhow!(e).context(format!("invalid UTF-8 in `{key}` header")),
                    StatusCode::BAD_REQUEST,
                )
            })
        })
        .transpose()
}

#[cfg(test)]
mod tests {
    use axum::http::{HeaderMap, HeaderValue};
    use risingwave_common::row::{OwnedRow, Row};
    use risingwave_common::types::{DataType, ScalarImpl, ToOwnedDatum};

    use super::*;

    fn decode_payload_row(columns: &[WebhookTableColumnDesc], payload: &[u8]) -> Result<OwnedRow> {
        let mut access_builder = build_json_access_builder(&HeaderMap::new())?;
        owned_row_from_payload_row(&mut access_builder, columns, payload)

View on GitHub (pinned to 6469eb736d)

Solutions

1. Inspect and re-send the offending header with plain ASCII/UTF-8 content.
2. Strip or replace non-ASCII characters from the value on the client side.
3. Check intermediate proxies/CDNs that may re-encode headers.
4. Avoid copy-pasting from rich-text editors that insert smart quotes.

Example fix

// before
headers.insert("x-rw-timestamp-handling", “utc”.parse().unwrap()); // smart quotes
// after
headers.insert("x-rw-timestamp-handling", "utc".parse().unwrap());

for (const [k, v] of Object.entries(headers)) {
  if (!/^\x20-\x7e*$/.test(v)) throw new Error(`header ${k} contains non-ASCII characters: ${JSON.stringify(v)}`);
}

// Rust
match response_or_result {
    Err(e) if e.to_string().contains("invalid UTF-8") => {
        // sanitize the offending header and retry once
    }
    other => other?,
}

When it happens

Trigger: Sending any webhook connector header (timestamp handling, timestamptz handling, time handling, bigint handling, boolean flags) containing non-ASCII or raw non-UTF-8 bytes, e.g. values with curly quotes, BOM, or binary garbage.

Common situations: Proxies or clients mangling header encodings, copy-pasted values containing smart quotes or zero-width characters, scripts writing latin-1 encoded headers.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• decode utf8 error: {0}
• InvalidInput
• Vault API returned error status: {} - {}
• Invalid UTF8 value encoding: {0}
• request error

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/f11c185b04f263d4. Report an issue: GitHub.