risingwavelabs/risingwave · error · SinkError::Config

HTTP sink with multiple columns only supports payload and ur

Error message

HTTP sink with multiple columns only supports payload and url columns, got {}

What it means

Schema validation in validate_http_sink: when the sink schema has more than one column, every column must be named `payload` or `url`; any other column name triggers this error. It names the offending column so the user can rename or drop it.

Source

Thrown at src/connector/src/sink/http.rs:141

    let fields = schema.fields();
    let (payload_index, url, payload_type) = if fields.len() == 1 {
        let Some(url) = url else {
            return Err(SinkError::Config(anyhow!(
                "HTTP sink requires url option when schema has exactly 1 column"
            )));
        };
        let url = url
            .parse()
            .context("invalid URL")
            .map_err(SinkError::Config)?;
        (0, HttpUrl::Static(url), fields[0].data_type.clone())
    } else {
        for field in fields {
            match field.name.as_str() {
                HTTP_SINK_PAYLOAD_COLUMN | HTTP_SINK_URL_COLUMN => {}
                _ => {
                    return Err(SinkError::Config(anyhow!(
                        "HTTP sink with multiple columns only supports payload and url columns, got {}",
                        field.name
                    )));
                }
            }
        }

        let payload_index = fields
            .iter()
            .position(|field| field.name == HTTP_SINK_PAYLOAD_COLUMN)
            .ok_or_else(|| {
                SinkError::Config(anyhow!(
                    "HTTP sink with multiple columns requires a payload column"
                ))
            })?;
        let url_index = fields
            .iter()
            .position(|field| field.name == HTTP_SINK_URL_COLUMN);

View on GitHub (pinned to 6469eb736d)

Solutions

1. Restrict the MV to only payload and url columns (project explicitly in SELECT)
2. Rename your columns to the expected payload/url column names
3. Collapse extra data into the payload column (e.g. JSON-encode it)

Example fix

// before
CREATE MATERIALIZED VIEW mv AS SELECT payload, url, extra_meta FROM src;
// after
CREATE MATERIALIZED VIEW mv AS SELECT payload, url FROM src;

// verify only allowed columns exist for multi-column schemas
let allowed: Vec<&str> = vec!["payload", "url"]; // per HTTP_SINK_* constants
let names: Vec<&str> = schema.fields().iter().map(|f| f.name.as_str()).collect();
if names.len() > 1 && names.iter().any(|n| !allowed.contains(n)) {
    return Err(format!("unsupported columns: only payload/url allowed, got {names:?}"));
}

match sink::try_create(props) {
    Err(e) if e.to_string().contains("payload and url columns") => {
        log::error!("project the MV down to payload/url columns: {e}");
    }
    r => r?,
}

When it happens

Trigger: `validate_http_sink` iterates multi-column fields and finds a name that is neither HTTP_SINK_PAYLOAD_COLUMN nor HTTP_SINK_URL_COLUMN.

Common situations: User tries to send extra metadata columns alongside the payload; schema of the MV inherited extra columns from SELECT *; renamed payload/url columns.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• HTTP sink with multiple columns requires a payload column
• Columns mismatch. RisingWave schema has {} fields, DeltaLake
• column {} not found in deltalake table
• Can't find data
• HTTP sink method must be POST or PUT, got '{method}'

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/19706960a85d5ee6. Report an issue: GitHub.