specified backfill parallelism {n} should not exceed max par

Error message

specified backfill parallelism {n} should not exceed max parallelism {max_parallelism}

What it means

`reschedule_backfill_parallelism_inplace` applies a backfill `ParallelismPolicy` to a streaming job. If the policy sets `StreamingParallelism::Fixed(n)` with `n` greater than the job's `max_parallelism`, the change is rejected with this error. It enforces that backfill parallelism stays within the job's upper bound.

Source

Thrown at src/meta/src/stream/scale.rs:307

        let txn = inner.db.begin().await?;

        for (table_id, policy) in &policy {
            let streaming_job = StreamingJob::find_by_id(*table_id)
                .one(&txn)
                .await?
                .ok_or_else(|| MetaError::catalog_id_not_found("table", table_id))?;

            let max_parallelism = streaming_job.max_parallelism;

            let mut streaming_job = streaming_job.into_active_model();

            if let Some(ParallelismPolicy {
                parallelism: StreamingParallelism::Fixed(n),
                ..
            }) = policy
                && *n > max_parallelism as usize
            {
                bail!(format!(
                    "specified backfill parallelism {n} should not exceed max parallelism {max_parallelism}"
                ));
            }

            if let Some(policy) = policy {
                streaming_job.backfill_parallelism = Set(Some(policy.parallelism.clone()));
                if matches!(policy.parallelism, StreamingParallelism::Fixed(_)) {
                    streaming_job.backfill_adaptive_parallelism_strategy = Set(None);
                } else {
                    streaming_job.backfill_adaptive_parallelism_strategy =
                        Set(policy.adaptive_parallelism_strategy.clone());
                }
            } else {
                streaming_job.backfill_parallelism = Set(None);
                streaming_job.backfill_adaptive_parallelism_strategy = Set(None);
            }
            streaming_job.update(&txn).await?;
        }

View on GitHub (pinned to 6469eb736d)

Solutions

Set backfill parallelism to a value <= the job's max_parallelism (check the catalog/SHOW output first).
Use Adaptive backfill parallelism instead of a fixed value.
If a larger fixed value is required, recreate the job with higher max_parallelism first.

Example fix

// before
ALTER TABLE t SET BACKFILL_PARALLELISM = 32; -- max_parallelism is 12
// after
ALTER TABLE t SET BACKFILL_PARALLELISM = 12;

Defensive patterns

Strategy: validation

Validate before calling

let max = get_job_max_parallelism(job_id).await?;
if let Some(policy) = &backfill_policy {
    if let StreamingParallelism::Fixed(n) = policy.parallelism {
        if n > max { return Err(format!("backfill {n} exceeds max {max}")); }
    }
}

Try / catch

match set_backfill_parallelism(job_id, policy).await {
    Err(e) if e.to_string().contains("backfill parallelism") => {
        set_backfill_parallelism(job_id, policy.with_fixed(max_parallelism)).await?
    }
    r => r?,
}

Prevention

Cap requested backfill parallelism at the job's max_parallelism.
Use Adaptive backfill parallelism unless a specific fixed value is required.
Remember backfill parallelism is bounded by job max_parallelism, not cluster size.

When it happens

Trigger: Calling the reschedule backfill parallelism API (e.g. `ALTER ... SET BACKFILL_PARALLELISM = n` or the corresponding `alter_streaming_job_backfill_parallelism` path) with a fixed `n > max_parallelism` of the job.

Common situations: Tuning backfill speed after a snapshot migration and picking a value above the job's max; misunderstanding that backfill parallelism is capped by job max_parallelism; copying settings from a job with a higher limit.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/b6c8e3dd6d24e3fe. Report an issue: GitHub.