risingwavelabs/risingwave · error

invalid backfill state: row_count

Error message

invalid backfill state: row_count

What it means

The CDC backfill state row follows the schema `| split_id | pk... | backfill_finished | row_count | cdc_offset |`. `restore_state` requires `row_count` (second-to-last column) to be a non-NULL Int64; anything else means the persisted backfill state is invalid and recovery aborts with this error.

Source

Thrown at src/stream/src/executor/backfill/cdc/state.rs:74

    }

    /// Restore the backfill state from storage
    pub async fn restore_state(&mut self) -> StreamExecutorResult<CdcStateRecord> {
        let key = Some(self.split_id.clone());
        match self
            .state_table
            .get_row(row::once(key.map(ScalarImpl::from)))
            .await?
        {
            Some(row) => {
                tracing::info!("restored cdc backfill state: {:?}", row);
                self.cached_state = row.into_inner().into_vec();
                let state = self.cached_state.as_slice();
                let state_len = state.len();
                // schema: | `split_id` | `pk...` | `backfill_finished` | `row_count` | `cdc_offset` |
                let row_count = match state[state_len - 2] {
                    Some(ScalarImpl::Int64(val)) => val,
                    _ => return Err(anyhow!("invalid backfill state: row_count").into()),
                };
                let is_finished = match state[state_len - 3] {
                    Some(ScalarImpl::Bool(val)) => val,
                    _ => return Err(anyhow!("invalid backfill state: backfill_finished").into()),
                };
                let cdc_offset = match state[state_len - 1] {
                    Some(ScalarImpl::Jsonb(ref jsonb)) => {
                        serde_json::from_value(jsonb.clone().take()).unwrap()
                    }
                    None if is_finished => None,
                    None => {
                        return Err(anyhow!(
                            "invalid backfill state: unfinished row has null cdc_offset"
                        )
                        .into());
                    }
                    _ => return Err(anyhow!("invalid backfill state: cdc_offset").into()),
                };

View on GitHub (pinned to 6469eb736d)

Solutions

1. Drop and re-create the CDC table (or the materialized/source using CDC backfill) to rebuild state from scratch.
2. Confirm writer and reader RisingWave versions match; if an upgrade happened, rebuild rather than recover.
3. Inspect the state row (schema shown in the error context) to confirm which column is malformed.
4. Restore the state table from a known-good backup/snapshot if available.

When it happens

Trigger: Recovering a CDC backfill executor whose persisted state row has a NULL or non-Int64 `row_count` column — incomplete state write before a crash, schema drift, or corrupted state table.

Common situations: Crash during CDC backfill state commit, RisingWave upgrades changing state schema, or manually modified/corrupted state store rows.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• invalid backfill state: backfill_finished
• invalid backfill state: unfinished row has null cdc_offset
• ParallelizedCdcBackfillExecutor expects either Mutation::Add
• legacy no-shuffle backfill recovered unfinished progress; ca
• implement MySQL CDC parallelized backfill

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/666ca72779ef3996. Report an issue: GitHub.