{"record":{"id":"a1174e815cc32dac","repo":"risingwavelabs/risingwave","slug":"should-not-update-vnode-bitmap-during-consuming-lo","errorCode":null,"errorMessage":"should not update vnode bitmap during consuming log store","messagePattern":"should not update vnode bitmap during consuming log store","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/stream/src/executor/backfill/snapshot_backfill/executor.rs","lineNumber":428,"sourceCode":"                                if is_finished {\n                                    assert_eq!(upstream_buffer.pending_epoch_lag(), 0);\n                                    assert!(pending_non_checkpoint_barrier.is_empty());\n                                    self.progress.finish_consuming_log_store(barrier.epoch);\n                                } else {\n                                    self.progress.update_create_mview_log_store_progress(\n                                        barrier.epoch,\n                                        upstream_buffer.pending_epoch_lag(),\n                                    );\n                                }\n                            }\n\n                            let post_commit = backfill_state.commit(barrier.epoch).await?;\n                            let update_vnode_bitmap =\n                                barrier.as_update_vnode_bitmap(self.actor_ctx.id);\n                            yield Message::Barrier(barrier);\n                            post_commit.post_yield_barrier(None).await?;\n                            if update_vnode_bitmap.is_some() {\n                                return Err(anyhow!(\n                                    \"should not update vnode bitmap during consuming log store\"\n                                )\n                                .into());\n                            }\n\n                            if is_finished {\n                                assert!(\n                                    pending_non_checkpoint_barrier.is_empty(),\n                                    \"{pending_non_checkpoint_barrier:?}\"\n                                );\n                                break;\n                            }\n                        }\n                        trace!(\n                            ?barrier_epoch,\n                            table_id = %self.upstream_table.table_id(),\n                            \"finish consuming log store\"\n                        );","sourceCodeStart":410,"sourceCodeEnd":446,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/stream/src/executor/backfill/snapshot_backfill/executor.rs#L410-L446","documentation":"While the backfill executor is in the log-store-consuming phase (after snapshot backfill finishes), it commits each barrier. An update-vnode-bitbarrier in this phase would invalidate the ongoing consumption, which the design does not support, so the executor returns this invariant error.","triggerScenarios":"A barrier carrying an update_vnode_bitmap (from a cluster scale-in/out or upstream parallelism change) arrives while this actor is consuming the log store phase of snapshot backfill.","commonSituations":"Rescaling the cluster (adding/removing nodes) during the log-store phase of a snapshot backfill; upstream parallelism change mid-backfill.","solutions":["Wait until backfill completes before scaling the cluster; retry the job after rescheduling with a stable topology","Cancel and recreate the streaming job so backfill runs under the new vnode assignment from the start","Check meta logs for the scale-in/out command that triggered the vnode update and reschedule it after backfill","If frequent, upgrade to a version supporting rescheduling during backfill (shuffled backfill)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Guard at the operational level: defer scale-in/out while backfill is in the log-store phase\nif backfill_in_progress && pending_scale_command { postpone_scale(); }","typeGuard":null,"tryCatchPattern":"if let Err(e) = run_backfill().await {\n    if e.to_string().contains(\"update vnode bitmap during consuming log store\") {\n        // recreate the job so backfill runs under the new vnode layout\n    }\n}","preventionTips":["Schedule cluster rescaling outside backfill windows","Prefer the shuffled/snapshot backfill executor, which supports rescheduling, over legacy paths"],"tags":["rust","streaming","backfill","vnode","scaling"],"backgroundTag":"invalid-state-transition","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}