risingwavelabs/risingwave · error

downstream relation missing for

Error message

downstream relation missing for {} -> {}

What it means

Thrown by build_reschedule_commands during rescale when a downstream relation entry expected in the `downstream_relations` map is absent for a (fragment_id, downstream_fragment_id) pair. The map is built from the previous fragment graph, so its absence means the plan was not assembled with all upstream->downstream edges intact. It is an internal consistency check in the scale (parallelism/rewrite) path.

Solutions

1. Retry the rescale operation once other DDL on the job has settled
2. Check meta logs for concurrent DDL on the same table/mv during rescale
3. Verify the job's fragment graph is intact via RW_FRAGMENT_STATS / meta node metadata
4. Upgrade to a version with the fix if this reproduces deterministically

Example fix

// No caller-side code fix; retry rescale after no other DDL is running
// before: ALTER TABLE t SET PARALLELISM 4;  (while CREATE MV on t is running)
// after:  wait for CREATE MV to finish, then ALTER TABLE t SET PARALLELISM 4;

-- before rescaling, ensure no other DDL is running on the job
SELECT * FROM rw_catalog.rw_ddl_progress WHERE table_name = 'my_mv';

match rescale_result {
    Err(e) if e.to_string().contains("downstream relation missing") => {
        // wait for concurrent DDL, then retry rescale
    }
    r => r?,
}

When it happens

Trigger: Rescale (ALTER ... SET PARALLELISM / pause-resume migration) where a downstream fragment lookup key is missing because the edge was already removed by an earlier iteration, or the reschedule context omitted the edge.

Common situations: Running rescale on a streaming job whose fragment graph changed concurrently (another DDL in flight), or a meta-service bug/state corruption after partial migration.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• should not update vnode bitmap during consuming log store
• Actor exited unexpectedly
• Array error
• below watermark check condition eval must return bool array
• cannot find StreamActor of actor

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/3046ab48bd5b391a. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/src/stream/scale.rs:930

                            .collect(),
                    };

                let target_fragment_distribution = *distribution_type;

                let fragment_relation::Model {
                    source_fragment_id: _,
                    target_fragment_id: _,
                    dispatcher_type,
                    dist_key_indices,
                    output_indices,
                    output_type_mapping,
                } = downstream_relations
                    .remove(&(
                        *fragment_id as FragmentId,
                        *downstream_fragment_id as FragmentId,
                    ))
                    .ok_or_else(|| {
                        MetaError::from(anyhow!(
                            "downstream relation missing for {} -> {}",
                            fragment_id,
                            downstream_fragment_id
                        ))
                    })?;

                let pb_mapping = PbDispatchOutputMapping {
                    indices: output_indices.into_u32_array(),
                    types: output_type_mapping.unwrap_or_default().to_protobuf(),
                };

                let (dispatchers, _) = compose_dispatchers(
                    *distribution_type,
                    &source_fragment_actors,
                    *downstream_fragment_id,
                    target_fragment_distribution,
                    &target_fragment_actors,
                    dispatcher_type,

View on GitHub (pinned to 6469eb736d)