risingwavelabs/risingwave · error · anyhow::Error

batch refresh job {} has no snapshot backfill info

Error message

batch refresh job {} has no snapshot backfill info

What it means

During `inject_database_initial_barrier`, for a batch refresh job the meta service computes snapshot backfill info from the job's fragment graph, then `.ok_or_else` fails when no fragment carries snapshot-backfill state. It is an internal invariant: a batch refresh job must have exactly one snapshot backfill node, so its absence is a corrupted/incomplete job description.

Source

Thrown at src/meta/src/barrier/rpc.rs:744

                                    )
                                })
                                .collect(),
                        )
                    })
            })
            .collect();

        // Batch-refresh jobs are rendered outside `jobs`, but their upstream tables
        // must still start with log-store-enabled subscribers after recovery.
        for (job_id, render_result) in &batch_refresh {
            let snapshot_backfill_info = StreamFragmentGraph::collect_snapshot_backfill_info_impl(
                render_result
                    .fragment_infos
                    .values()
                    .map(|fragment| (&fragment.nodes, fragment.fragment_type_mask)),
            )?
            .0
            .ok_or_else(|| anyhow!("batch refresh job {} has no snapshot backfill info", job_id))?;

            for upstream_table_id in snapshot_backfill_info
                .upstream_mv_table_id_to_backfill_epoch
                .keys()
            {
                subscribers
                    .entry(*upstream_table_id)
                    .or_default()
                    .try_insert(job_id.as_subscriber_id(), SubscriberType::SnapshotBackfill)
                    .expect("non-duplicate");
            }
        }

        let mut database_jobs = HashMap::new();
        let mut snapshot_backfill_jobs = HashMap::new();

        for (job_id, job_fragments) in jobs {
            if creating_jobs.remove(&job_id) {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Inspect the job's fragment metadata in the meta store; confirm the snapshot backfill fragment exists and its type mask is set.
2. Re-create or re-issue the batch refresh job if its fragment graph is inconsistent.
3. Check for version skew: ensure meta nodes and frontend run versions that set snapshot backfill info when creating batch refresh jobs.
4. File a bug with the job_id and fragment dump if a freshly created job hits this.

// Before injecting, assert the job has snapshot backfill info
fn has_snapshot_backfill_info(fragment_infos: &BTreeMap<u32, StreamFragmentInfo>) -> bool {
    fragment_infos.values().any(|f| {
        f.nodes.values().any(|n| {
            n.node.as_ref()
                .map(|n| n.snapshot_backfill_info.is_some())
                .unwrap_or(false)
        })
    })
}

When it happens

Trigger: `inject_database_initial_barrier` processing a batch refresh job whose resolved fragment graph yields no snapshot backfill info (fragment type mask lookup returns None).

Common situations: Metadata inconsistency after partial recovery or upgrade; job created/modified by code path that didn't register the snapshot backfill node; corrupt object store state for the job's fragments.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• recovered snapshot backfill job {} has no snapshot backfill
• should not pause when having snapshot backfill job {job_id}
• database of cross db upstream table {} not found
• cannot reschedule jobs {:?} when creating jobs with unresche
• cannot create streaming job with snapshot backfill when paus

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/14bc9b8f80e471ca. Report an issue: GitHub.