{"record":{"id":"f223316b6872918a","repo":"risingwavelabs/risingwave","slug":"unexpected-backfill-state","errorCode":null,"errorMessage":"Unexpected backfill state: {:?}","messagePattern":"Unexpected backfill state: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/stream/src/executor/source/source_backfill_executor.rs","lineNumber":1167,"sourceCode":"        // Checks added splits\n        for split_id in &target_splits {\n            if !states.contains_key(split_id) {\n                split_changed = true;\n\n                let backfill_state = committed_reader\n                    .try_recover_from_state_store(split_id)\n                    .await?;\n                match &backfill_state {\n                    None => {\n                        // Newly added split. We don't need to backfill!\n                        newly_added_splits.push(split_id.clone());\n                    }\n                    Some(backfill_state) => {\n                        // Migrated split. It should also be finished since we are in forwarding stage.\n                        match backfill_state.state {\n                            BackfillState::Finished => {}\n                            _ => {\n                                return Err(anyhow::anyhow!(\n                                    \"Unexpected backfill state: {:?}\",\n                                    backfill_state\n                                )\n                                .into());\n                            }\n                        }\n                    }\n                }\n                states.insert(\n                    split_id.clone(),\n                    backfill_state.unwrap_or(BackfillStateWithProgress {\n                        state: BackfillState::Finished,\n                        num_consumed_rows: 0,\n                        target_offset: None,\n                    }),\n                );\n            }\n        }","sourceCodeStart":1149,"sourceCodeEnd":1185,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/stream/src/executor/source/source_backfill_executor.rs#L1149-L1185","documentation":"During a split-change (split migration) handled in the forwarding stage, the source backfill executor expects every migrated split's backfill state to be Finished. Any other BackfillState means an inconsistent internal state machine, so update_state_if_changed_forward_stage fails with this error.","triggerScenarios":"apply_split_change_forward_stage_after_yield_barrier encounters a split whose backfill_state.state is not BackfillState::Finished while processing the forwarding stage after a split change.","commonSituations":"Split change raced with backfill completion, state corrupted or partially persisted across recovery, or a bug/edge case in the backfill state machine with parallelism changes.","solutions":["Capture the full backfill state from the error message and file an issue — this indicates an internal invariant violation.","Retry the MV after recovery; if it persists, recreate the materialized view to reset backfill state.","Check recent parallelism/split change operations on the source and whether they occurred during backfill."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"// before split changes, confirm all splits report backfill finished:\n// query backfill progress/state from system catalogs or metrics before rescaling","typeGuard":"fn is_finished(state: &BackfillState) -> bool {\n    matches!(state.state, BackfillState::Finished)\n}","tryCatchPattern":"match res.backfill_state {\n    Some(s) if matches!(s.state, BackfillState::Finished) => proceed(),\n    Some(s) => report_invariant_violation(format!(\"Unexpected backfill state: {:?}\", s)),\n    None => proceed(),\n}","preventionTips":["Avoid changing parallelism/splits while backfill is still in progress","Monitor backfill completion before issuing split-change commands","Report this error upstream: it signals an internal state machine bug"],"tags":["backfill","state-machine","internal-error"],"backgroundTag":"invalid-state-transition","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}