risingwavelabs/risingwave · error

progress table columns len not matched with the len derived

Error message

progress table columns len not matched with the len derived from upstream table pk. progress table: {:?}, pk: {:?}

What it means

The backfill progress (state) table's column count does not match the expected count: the upstream table's primary key columns plus the fixed extra columns (`EXTRA_COLUMN_TYPES`). The progress table must mirror this schema exactly to record per-vnode progress.

Source

Thrown at src/stream/src/executor/backfill/snapshot_backfill/state.rs:59

}

/// `vnode`, `epoch`, `row_count`, `is_finished`
const EXTRA_COLUMN_TYPES: [DataType; 4] = [
    DataType::Int16,
    DataType::Int64,
    DataType::Int64,
    DataType::Boolean,
];

impl VnodeBackfillProgress {
    fn validate_progress_table_schema(
        progress_table_column_types: &[DataType],
        upstream_pk_column_types: &[DataType],
    ) -> StreamExecutorResult<()> {
        if progress_table_column_types.len()
            != EXTRA_COLUMN_TYPES.len() + upstream_pk_column_types.len()
        {
            return Err(anyhow!(
                "progress table columns len not matched with the len derived from upstream table pk. progress table: {:?}, pk: {:?}",
                progress_table_column_types,
                upstream_pk_column_types)
                .into()
            );
        }
        for (expected_type, progress_table_type) in EXTRA_COLUMN_TYPES
            .iter()
            .chain(upstream_pk_column_types.iter())
            .zip_eq_debug(progress_table_column_types.iter())
        {
            if expected_type != progress_table_type {
                return Err(anyhow!(
                    "progress table column not matched with upstream table schema: progress table: {:?}, pk: {:?}",
                    progress_table_column_types,
                    upstream_pk_column_types)
                    .into()
                );

View on GitHub (pinned to 6469eb736d)

Solutions

1. Recreate the materialized view/table so the progress table is regenerated with the current expected schema.
2. Check RisingWave version compatibility of persisted progress tables before/after upgrade (see release notes for backfill state format changes).
3. Compare the progress table definition with the upstream table PK columns and fix whichever was changed unexpectedly.

assert_eq!(progress_table.column_types().len(), EXTRA_COLUMN_TYPES.len() + upstream_pk.len(), "progress table schema mismatch");

if let Err(e) = BackfillState::validate_progress_table_schema(&pt, &pk) { recreate_backfill_state(); return Err(e); }

When it happens

Trigger: `validate_progress_table_schema` is called with a progress table created by a different schema version than the one derived from the current upstream table PK — e.g. the upstream table's PK/distribution changed, or the progress table was created by an older RisingWave version.

Common situations: Upgrading a cluster across versions where the progress table schema changed; manually altering the upstream table's primary key; restoring a progress table from a mismatched source.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• schema invalid, record type required at top level of the sch
• PostgreSQL schema `{schema}` does not exist
• Invalid option: {message}
• Cannot find table in bigquery
• The length of the RisingWave column {} must be equal to the

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/abe774f1cb930aa9. Report an issue: GitHub.