risingwavelabs/risingwave · error

required column should be kept

Error message

required column should be kept

What it means

During column pruning of LogicalExpand, the mapping (`out_col_change`) produced by `rewrite_with_input` cannot map back one of the required output column indexes. The code asserts every required column survives the rewrite; if it does not, this `.expect` message panics, indicating an internal invariant violation in the expand-node column mapping rather than a user-facing failure.

Source

Thrown at src/frontend/src/optimizer/plan_node/logical_expand.rs:156

            if idx < input_len {
                input_required_cols.insert(idx);
            } else if idx < input_len * 2 {
                input_required_cols.insert(idx - input_len);
            } else {
                assert_eq!(idx, input_len * 2);
            }
        }
        let input_required_cols = input_required_cols.ones().collect_vec();
        let input_col_change =
            ColIndexMapping::with_remaining_columns(&input_required_cols, input_len);
        let input = self.input().prune_col(&input_required_cols, ctx);
        let (expand, out_col_change) = self.rewrite_with_input(input, input_col_change);
        let output_required_cols = required_cols
            .iter()
            .map(|idx| {
                out_col_change
                    .try_map(*idx)
                    .expect("required column should be kept")
            })
            .collect_vec();
        LogicalProject::with_out_col_idx(expand.into(), output_required_cols.into_iter()).into()
    }
}

impl ExprRewritable<Logical> for LogicalExpand {}

impl ExprVisitable for LogicalExpand {}

impl PredicatePushdown for LogicalExpand {
    fn predicate_pushdown(
        &self,
        predicate: Condition,
        ctx: &mut PredicatePushdownContext,
    ) -> PlanRef {
        // No pushdown.
        gen_filter_and_pushdown(self, predicate, Condition::true_cond(), ctx)

View on GitHub (pinned to 6469eb736d)

Solutions

1. Inspect the Expand node's mapping construction in `rewrite_with_input` to ensure all original output columns are preserved in `out_col_change`.
2. Reduce the failing query to a minimal GROUPING SETS/ROLLUP example and file a bug with the plan dump.
3. Check for recent changes to column pruning or Expand and re-run `./risedev c` plus planner tests after fixing the mapping.

// sanity-check mapping before pruning
for &idx in required_cols {
    assert!(
        out_col_change.try_map(idx).is_some(),
        "required col {idx} lost in expand rewrite"
    );
}

When it happens

Trigger: Calling `prune_col` on a LogicalExpand node where `out_col_change.try_map(idx)` returns None for a required column — i.e. the expand rewrite dropped a column the parent still requires.

Common situations: Optimizer bug reports/CI failures while running column pruning over queries with GROUPING SETS/ROLLUP/CUBE (which lower to LogicalExpand), usually after changes to Expand or pruning logic.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• iceberg intermediate scan must have a source catalog
• output column must exist in the source catalog
• call prune_col of the PlanRef instead of calling directly on
• unreachable
• internal error: entered unreachable code

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/2e912c0a83368f03. Report an issue: GitHub.