risingwavelabs/risingwave · error · SinkError::Config

HTTP sink with multiple columns requires a payload column

Error message

HTTP sink with multiple columns requires a payload column

What it means

Schema validation in validate_http_sink for multi-column HTTP sinks: after confirming all columns are `payload` or `url`, a column actually named `payload` must exist to carry the request body. It fires when a multi-column schema has only a `url` column (or none of the allowed names' payload counterpart), leaving the HTTP request body undefined.

Source

Thrown at src/connector/src/sink/http.rs:153

        (0, HttpUrl::Static(url), fields[0].data_type.clone())
    } else {
        for field in fields {
            match field.name.as_str() {
                HTTP_SINK_PAYLOAD_COLUMN | HTTP_SINK_URL_COLUMN => {}
                _ => {
                    return Err(SinkError::Config(anyhow!(
                        "HTTP sink with multiple columns only supports payload and url columns, got {}",
                        field.name
                    )));
                }
            }
        }

        let payload_index = fields
            .iter()
            .position(|field| field.name == HTTP_SINK_PAYLOAD_COLUMN)
            .ok_or_else(|| {
                SinkError::Config(anyhow!(
                    "HTTP sink with multiple columns requires a payload column"
                ))
            })?;
        let url_index = fields
            .iter()
            .position(|field| field.name == HTTP_SINK_URL_COLUMN);
        let url = match (url, url_index) {
            (Some(_), Some(_)) => {
                return Err(SinkError::Config(anyhow!(
                    "HTTP sink url option cannot coexist with url column"
                )));
            }
            (Some(url), None) => {
                let url = url
                    .parse()
                    .context("invalid URL")
                    .map_err(SinkError::Config)?;
                HttpUrl::Static(url)

View on GitHub (pinned to 6469eb736d)

Solutions

1. Add/rename a column to the expected payload column name
2. Ensure your MV projects both payload (and optionally url) columns
3. Check exact column-name spelling per HTTP sink docs

Example fix

// before
CREATE MATERIALIZED VIEW mv AS SELECT target_url FROM src;
// after
CREATE MATERIALIZED VIEW mv AS SELECT payload, target_url FROM src;

// ensure the payload column exists in multi-column schemas
let names: Vec<&str> = schema.fields().iter().map(|f| f.name.as_str()).collect();
if names.len() > 1 && !names.contains(&"payload") {
    return Err("multi-column HTTP sink schema requires a payload column".into());
}

match sink::try_create(props) {
    Err(e) if e.to_string().contains("requires a payload column") => {
        log::error!("add the payload column to the MV: {e}");
    }
    r => r?,
}

When it happens

Trigger: `validate_http_sink` finds fields.iter().position(payload column) == None for a multi-column schema.

Common situations: User defines only a url column expecting GET-like behavior; payload column misspelled; schema projection dropped the payload column.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• HTTP sink with multiple columns only supports payload and ur
• Can't find data
• HTTP sink method must be POST or PUT, got '{method}'
• HTTP sink requires url option when schema has exactly 1 colu
• schema invalid, record type required at top level of the sch

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/b44ab619ad0ed60c. Report an issue: GitHub.