{"record":{"id":"347d4084ca005fbf","repo":"risingwavelabs/risingwave","slug":"invalid-backfill-state-backfill-finished","errorCode":null,"errorMessage":"invalid backfill state: backfill_finished","messagePattern":"invalid backfill state: backfill_finished","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/stream/src/executor/backfill/cdc/state.rs","lineNumber":78,"sourceCode":"        let key = Some(self.split_id.clone());\n        match self\n            .state_table\n            .get_row(row::once(key.map(ScalarImpl::from)))\n            .await?\n        {\n            Some(row) => {\n                tracing::info!(\"restored cdc backfill state: {:?}\", row);\n                self.cached_state = row.into_inner().into_vec();\n                let state = self.cached_state.as_slice();\n                let state_len = state.len();\n                // schema: | `split_id` | `pk...` | `backfill_finished` | `row_count` | `cdc_offset` |\n                let row_count = match state[state_len - 2] {\n                    Some(ScalarImpl::Int64(val)) => val,\n                    _ => return Err(anyhow!(\"invalid backfill state: row_count\").into()),\n                };\n                let is_finished = match state[state_len - 3] {\n                    Some(ScalarImpl::Bool(val)) => val,\n                    _ => return Err(anyhow!(\"invalid backfill state: backfill_finished\").into()),\n                };\n                let cdc_offset = match state[state_len - 1] {\n                    Some(ScalarImpl::Jsonb(ref jsonb)) => {\n                        serde_json::from_value(jsonb.clone().take()).unwrap()\n                    }\n                    None if is_finished => None,\n                    None => {\n                        return Err(anyhow!(\n                            \"invalid backfill state: unfinished row has null cdc_offset\"\n                        )\n                        .into());\n                    }\n                    _ => return Err(anyhow!(\"invalid backfill state: cdc_offset\").into()),\n                };\n\n                let current_pk_pos = state[1..state_len - 3].to_vec();\n                Ok(CdcStateRecord {\n                    current_pk_pos: Some(OwnedRow::new(current_pk_pos)),","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/stream/src/executor/backfill/cdc/state.rs#L60-L96","documentation":"Same CDC backfill state schema; `restore_state` requires the `backfill_finished` column (third from the end) to be a non-NULL Bool. A NULL or non-bool value means the persisted state cannot be interpreted, so recovery bails with this error.","triggerScenarios":"Recovery of a CDC backfill executor when the state row's `backfill_finished` slot is NULL or holds a non-boolean scalar — incomplete write before crash, schema drift, or corruption.","commonSituations":"Crash between partial state writes during CDC backfill, upgrades with changed state layout, or corrupted Hummock state rows.","solutions":["Re-create the CDC table / backfilled object so its state table is rebuilt from the beginning.","Check version compatibility between the build that wrote the state and the current one; rebuild after upgrades.","Dump the state row and verify the column layout matches `split_id | pk | backfill_finished | row_count | cdc_offset`.","Restore from backup of the state store if the data is valuable and corruption is confirmed."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":["Rebuild CDC backfill state (drop/re-create table) after version upgrades.","Avoid abrupt compute-node kills during CDC backfill; drain via proper shutdown.","Validate state schema assumptions after any manual state-store surgery."],"tags":["streaming","cdc","backfill","state","recovery"],"backgroundTag":"schema-validation-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}