ParallelizedCdcBackfillExecutor expects either Mutation::Add

Error message

ParallelizedCdcBackfillExecutor expects either Mutation::Add or Mutation::Update to initialize CDC table snapshot splits.

What it means

`ParallelizedCdcBackfillExecutor::execute_inner` expects the incoming reset barrier's mutation to be `Mutation::Add` or `Mutation::Update`, both of which carry CDC table snapshot splits used to initialize backfill. Any other mutation kind (or none) on a reset barrier is invalid initialization input, so it returns this error.

Source

Thrown at src/stream/src/executor/backfill/cdc/cdc_backill_v2.rs:167

            handle_toast_columns,
        )
        .boxed();
        let mut next_reset_barrier = Some(first_barrier);
        let mut is_reset = false;
        let mut state_impl = ParallelizedCdcBackfillState::new(self.state_table);
        // The buffered chunks have already been mapped.
        let mut upstream_chunk_buffer: Vec<StreamChunk> = vec![];

        // Need reset on CDC table snapshot splits reschedule.
        'with_cdc_table_snapshot_splits: loop {
            assert!(upstream_chunk_buffer.is_empty());
            let reset_barrier = next_reset_barrier.take().unwrap();
            let all_snapshot_splits = match reset_barrier.mutation.as_deref() {
                Some(Mutation::Add(add)) => &add.actor_cdc_table_snapshot_splits.splits,

                Some(Mutation::Update(update)) => &update.actor_cdc_table_snapshot_splits.splits,
                _ => {
                    return Err(anyhow::anyhow!("ParallelizedCdcBackfillExecutor expects either Mutation::Add or Mutation::Update to initialize CDC table snapshot splits.").into());
                }
            };
            let mut actor_snapshot_splits = vec![];
            let mut generation = None;
            // TODO(zw): optimization: remove consumed splits to reduce barrier size for downstream.
            if let Some((splits, snapshot_generation)) = all_snapshot_splits.get(&self.actor_ctx.id)
            {
                actor_snapshot_splits = splits
                    .iter()
                    .map(|s: &CdcTableSnapshotSplitRaw| {
                        let de = RowDeserializer::new(
                            cdc_table_snapshot_split_column
                                .iter()
                                .map(Field::data_type)
                                .collect_vec(),
                        );
                        let left_bound_inclusive =
                            de.deserialize(s.left_bound_inclusive.as_ref()).unwrap();

View on GitHub (pinned to 6469eb736d)

Solutions

Check meta logs around the CDC table creation/reschedule time to find which mutation was sent to this actor.
Retry the CDC table creation (drop and re-create the CDC table / CDC source) so a fresh, well-formed reset barrier is issued.
Ensure meta and compute node versions match; version skew can change mutation payload shapes.
If reproducible on one version, file with the barrier type observed; upgrade in case the rescheduler fix already landed.

Defensive patterns

Strategy: retry

Try / catch

// Wrap CDC table creation/retry loop:
match create_cdc_table_with_backfill().await {
    Err(e) if e.to_string().contains("expects either Mutation::Add or Mutation::Update") => {
        log::warn!("malformed reset barrier during CDC backfill; retrying creation");
        drop_cdc_table().await?;
        create_cdc_table_with_backfill().await?;
    }
    other => other?,
}

Prevention

Keep meta and compute nodes on identical versions so mutation payloads match.
Avoid rescheduling CDC backfill actors under heavy DDL load; check meta logs for reschedule storms.
Retry CDC table creation cleanly (drop then re-create) instead of forcing recovery.

When it happens

Trigger: A reset/merge barrier reaches the CDC backfill executor without the expected Add/Update mutation carrying `actor_cdc_table_snapshot_splits` — e.g. reschedule/merge logic sends a wrong mutation type, or the barrier was constructed without snapshot splits.

Common situations: Meta-node rescheduling bugs during CDC table backfill, recovery scenarios where the reset barrier is replayed with a different mutation, or version-skew between meta and compute nodes.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/eff855726348429c. Report an issue: GitHub.