risingwavelabs/risingwave · error

empty payload with non-empty key

Error message

empty payload with non-empty key

What it means

The only_parse_payload! macro bails when a Debezium/native upsert event carries no payload but does carry a key. In upsert semantics the payload (after/value) is required to know what to upsert; a key-only event cannot be interpreted, so parsing fails loudly.

Source

Thrown at src/connector/src/parser/utils.rs:91

        .error_for_status()
        .with_context(|| format!("http request failed for {location}"))?;

    let bytes = res
        .bytes()
        .await
        .with_context(|| format!("failed to read HTTP body of {location}"))?;

    Ok(bytes)
}

// For parser that doesn't support key currently
#[macro_export]
macro_rules! only_parse_payload {
    ($self:ident, $payload:ident, $writer:ident) => {
        if let Some(payload) = $payload {
            $self.parse_inner(payload, $writer).await
        } else {
            risingwave_common::bail!("empty payload with non-empty key")
        }
    };
}

/// Load raw bytes from:
/// * local file, for on-premise or testing.
/// * http/https, for common usage.
/// * s3 file location format: <s3://bucket_name/file_name>
pub(super) async fn bytes_from_url(
    url: &Url,
    config: Option<&AwsAuthProps>,
) -> ConnectorResult<Vec<u8>> {
    match (url.scheme(), config) {
        // TODO(Tao): support local file only when it's compiled in debug mode.
        ("file", _) => {
            let path = url
                .to_file_path()
                .ok()

View on GitHub (pinned to 6469eb736d)

Solutions

1. Ensure producers always emit a value payload for upsert records, or use the proper delete/tombstone format supported by RisingWave (explicit null value handled as DELETE).
2. Check the Debezium/Kafka producer config so compacted tombstones aren't consumed as key-only non-null-payload records.
3. If key-only events are expected, handle them before parse_inner instead of routing through only_parse_payload!.

Example fix

// before (producer)
producer.send(KeyRecord { key }, None); // key-only message
// after
producer.send(KeyRecord { key }, Some(ValueRecord { .. }));

// producer-side guard before emitting an upsert record
if value.is_none() && !is_tombstone_format_supported {
    return Err("cannot emit key-only record to upsert source".into());
}

match res {
    Err(e) if e.to_string().contains("empty payload with non-empty key") => {
        // inspect upstream topic for key-only/tombstone messages
    }
    Err(e) => return Err(e.into()),
    Ok(v) => v,
}

When it happens

Trigger: Kafka log-compacted tombstone-like messages in an upsert source where the message has a key but the value/payload field is absent rather than explicitly null-deleted; a parser invoking only_parse_payload! with None payload.

Common situations: Kafka compaction emitting messages with null values (tombstones) into an UPSERT source; producers writing key-only records; misconfigured Debezium serializer emitting empty value envelopes.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• Unsupported encoding for Plain
• Parquet parser error: {message}
• {0}
• avro parse unexpected eof
• protobuf key is not supported

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/386c4698dac3fde9. Report an issue: GitHub.