[TRANSFORM-AGGREGATOR] Invalid hash table state during…

Error message

[TRANSFORM-AGGREGATOR] Invalid hash table state during spill check

What it means

TransformPartialAggregate::spill_out rebuilds/reads the hash table for spilling and expects HashTable::AggregateHashTable; any other state (MovedOut) hits unreachable!("Invalid hash table state during spill check"). It asserts spills only occur while the partial aggregation table is still owned.

Solutions

Update to a fixed version and rerun the query
Ensure the pipeline invokes spill checks only before on_finish
File a bug with the stack trace and memory/spill settings if reproducible

Defensive patterns

Strategy: try-catch

Type guard

fn can_spill(ht: &HashTable) -> bool {
    matches!(ht, HashTable::AggregateHashTable(_))
}

Try / catch

match res {
    Err(e) if e.message().contains("Invalid hash table state during spill check") => {
        // retry; tune memory limits so spill happens mid-execution, not at finish
    }
    ...
}

Prevention

Verify spill checks only run during the transform phase, before on_finish
Include spill interleavings in executor tests
Upgrade promptly when hitting this in large aggregations
Avoid duplicate driver invocations of check_spill after finish

When it happens

Trigger: check_spill triggers spill_out after the table was taken/moved out — from finish running before spill checks complete, pipeline driver ordering bugs, or cancellation racing with spill.

Common situations: Large aggregation spilling to disk combined with early finish; executor state-machine regressions; duplicated transform invocations in custom pipelines.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/0d8cfe46f095ac18. Report an issue: GitHub.

Appendix: source

Thrown at src/query/service/src/pipelines/processors/transforms/aggregator/transform_aggregate_partial.rs:223

            }
        }
    }

    fn spill_out(&mut self) -> Result<()> {
        if let HashTable::AggregateHashTable(v) = std::mem::take(&mut self.hash_table) {
            let config = v.config.clone();

            self.spillers.spill(v.payload, self.is_row_shuffle)?;

            let arena = Arc::new(Bump::new());
            self.hash_table = HashTable::AggregateHashTable(AggregateHashTable::new(
                self.params.group_data_types.clone(),
                self.params.aggregate_functions.clone(),
                config,
                arena,
            ));
        } else {
            unreachable!("[TRANSFORM-AGGREGATOR] Invalid hash table state during spill check")
        }
        Ok(())
    }
}

impl AccumulatingTransform for TransformPartialAggregate {
    const NAME: &'static str = "TransformPartialAggregate";

    fn transform(&mut self, block: DataBlock) -> Result<Vec<DataBlock>> {
        self.execute_one_block(block)?;

        if self.settings.check_spill() {
            self.spill_out()?;
        }

        Ok(vec![])
    }

View on GitHub (pinned to 288d84d76e)