{"record":{"id":"c36bebe8cdedec02","repo":"risingwavelabs/risingwave","slug":"backfill-progress-for-vnode-not-found-backf","errorCode":null,"errorMessage":"Backfill progress for vnode {:#?} not found, backfill_state not initialized properly","messagePattern":"Backfill progress for vnode (.+?) not found, backfill_state not initialized properly","errorType":"error_code","errorClass":"StreamExecutorError","httpStatus":null,"severity":"error","filePath":"src/stream/src/executor/backfill/utils.rs","lineNumber":80,"sourceCode":"            )\n        })\n    }\n\n    pub(crate) fn get_current_state(\n        &mut self,\n        vnode: &VirtualNode,\n    ) -> &mut BackfillProgressPerVnode {\n        &mut self.inner.get_mut(vnode).unwrap().current_state\n    }\n\n    // Expects the vnode to always have progress, otherwise it will return an error.\n    pub(crate) fn get_progress(\n        &self,\n        vnode: &VirtualNode,\n    ) -> StreamExecutorResult<&BackfillProgressPerVnode> {\n        match self.inner.get(vnode) {\n            Some(p) => Ok(p.current_state()),\n            None => bail!(\n                \"Backfill progress for vnode {:#?} not found, backfill_state not initialized properly\",\n                vnode,\n            ),\n        }\n    }\n\n    pub(crate) fn update_progress(\n        &mut self,\n        vnode: VirtualNode,\n        new_pos: OwnedRow,\n        snapshot_row_count_delta: u64,\n    ) -> StreamExecutorResult<()> {\n        let state = self.get_current_state(&vnode);\n        match state {\n            BackfillProgressPerVnode::NotStarted => {\n                *state = BackfillProgressPerVnode::InProgress {\n                    current_pos: new_pos,\n                    snapshot_row_count: snapshot_row_count_delta,","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/stream/src/executor/backfill/utils.rs#L62-L98","documentation":"A lookup of backfill progress for a specific vnode found no entry in the `BackfillProgress` map. Every vnode the executor is responsible for must be initialized with progress state before chunks are read or marked; a missing entry means the state was never initialized from the progress table.","triggerScenarios":"`get_progress` is called (from `snapshot_read_per_vnode` or `mark_chunk_ref_by_vnode`) for a vnode absent from `self.inner`, e.g. state built from progress-table rows that don't cover all vnodes of the fragment, or vnode mapping changed after recovery.","commonSituations":"Progress table missing rows for some vnodes (partial write/corruption); changed parallelism/vnode mapping across restarts; code bug initializing state before first chunk arrives.","solutions":["Verify the progress table contains one row per vnode for this actor's vnode range and reinitialize missing entries from defaults.","Check whether vnode mapping (parallelism) changed since the progress table was written; migrate or recreate the backfill state.","Ensure the state is initialized via the init path before any snapshot read; if the bug is in initialization order, file an issue with the trace."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before reading/chunk-marking\nfor vnode in assigned_vnodes { debug_assert!(progress.contains(vnode), \"missing progress for {vnode:?}\"); }","typeGuard":"fn get_progress_safe<'a>(p: &'a BackfillProgress, vnode: &VirtualNode) -> Option<&'a BackfillProgressPerVnode> { p.inner.get(vnode).map(|e| e.current_state()) }","tryCatchPattern":"let progress = state.get_progress(&vnode).with_context(|| format!(\"vnode {vnode:?} missing; reinit state\"))?;","preventionTips":["Initialize progress entries for every vnode of the fragment before first snapshot read","Rebuild or migrate backfill state when parallelism/vnode mapping changes","Validate progress-table row coverage at recovery"],"tags":["backfill","state","vnode","not-found"],"backgroundTag":"record-not-found","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}