What causes "{connector_name} snapshot primary key `{}` cannot be NULL" in risingwavelabs/risingwave?

Fix the data: backfill or exclude NULL primary keys in the upstream table before snapshotting.

{connector_name} snapshot primary key `{}` cannot be NULL

Error message

{connector_name} snapshot primary key `{}` cannot be NULL

What it means

In strict primary-key decoding mode, `decode_row_with_strict_pk` decodes each field; non-PK decode failures are logged and yield NULL (lenient), but a NULL value for a primary-key column is fatal. This preserves the invariant that every decoded snapshot row has a non-NULL primary key, since RW rows must be keyed.

Source

Thrown at src/connector/src/parser/mod.rs:108

        let decode_result = decode(index, field);
        let datum = if is_pk {
            decode_result.with_context(|| {
                format!(
                    "failed to decode {connector_name} snapshot primary key `{}`",
                    field.name
                )
            })?
        } else {
            match decode_result {
                Ok(datum) => datum,
                Err(err) => {
                    log_non_pk_error(&field.name, err);
                    None
                }
            }
        };
        if is_pk && datum.is_none() {
            bail!(
                "{connector_name} snapshot primary key `{}` cannot be NULL",
                field.name
            );
        }
        datums.push(datum);
    }
    Ok(OwnedRow::new(datums))
}

#[cfg(test)]
mod strict_pk_tests {
    use anyhow::anyhow;
    use risingwave_common::row::Row;
    use risingwave_common::types::{DataType, ScalarImpl};

    use super::*;

    fn test_schema() -> Schema {

View on GitHub (pinned to 6469eb736d)

Solutions

Fix the data: backfill or exclude NULL primary keys in the upstream table before snapshotting.
Choose a non-nullable column (or composite key) as the RW primary key and update the table definition.
Investigate why the pk field decoded to None (check preceding decode-error logs from `log_non_pk_error` and the message schema) and fix the encoding/schema mapping.

Example fix

-- before: nullable column as pk
CREATE TABLE t (id INT PRIMARY KEY, ...); -- id is NULL in upstream
-- after: use a non-nullable key
CREATE TABLE t (uuid VARCHAR PRIMARY KEY, id INT, ...);

Defensive patterns

Strategy: validation

Validate before calling

if pk_columns.iter().any(|c| c.nullable) {
    return Err(anyhow!("primary key columns must be non-nullable for strict pk decoding"));
}

Try / catch

match decode_row_with_strict_pk(...) {
    Err(e) if e.to_string().contains("cannot be NULL") => {
        log::warn!("snapshot row with NULL pk skipped/quarantined: {e}");
    }
    other => other?,
}

Prevention

Only choose NOT NULL columns (or add a surrogate key) as RW primary keys for CDC tables.
Check for prior decode-error logs (log_non_pk_error) to distinguish a decode failure from a true NULL.
Validate upstream data for NULL keys before starting a snapshot.

When it happens

Trigger: A CDC snapshot record whose decoded primary-key field is absent, unparseable (non-PK-style decode produced None), or explicitly NULL is passed to `decode_row_with_strict_pk`.

Common situations: Upstream table has a nullable column used as RW primary key; the CDC payload omits the pk field; a decode failure on the pk column (type mismatch, encoding issue) silently produces None which then trips this check; schema drift so the value lands in the wrong column.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/9a09ec4e9ce93ba9. Report an issue: GitHub.