[TRANSFORM-AGGREGATOR] Invalid hash table state before spill

Error message

[TRANSFORM-AGGREGATOR] Invalid hash table state before spill

What it means

TransformFinalAggregate::finish, when spilled_occurred is true, reads statistics (payload length, resize count) from the hash table, requiring HashTable::AggregateHashTable. If the table is in another state (e.g. MovedOut), unreachable! fires, meaning the finish path executed after the hash table was already consumed.

Solutions

Retry on a fixed/updated version; ensure single invocation of the final transform's finish
Check custom pipeline event wiring so spill_out/finish ordering matches AccumulatingTransform contract
File a bug with stack trace if reproducible on a single version

Defensive patterns

Strategy: try-catch

Type guard

fn as_aggregate_ht(ht: &HashTable) -> Option<&AggregateHashTable> {
    match ht { HashTable::AggregateHashTable(t) => Some(t), _ => None }
}

Try / catch

match res {
    Err(e) if e.message().contains("Invalid hash table state before spill") => {
        // abort, retry on fixed version, capture stack trace
    }
    ...
}

Prevention

Ensure finish is invoked exactly once per transform instance
Test spill-then-finish ordering in pipeline integration tests
Avoid custom executors that double-drive AccumulatingTransform::process/finish

When it happens

Trigger: finish() is called with spilled_occurred set but self.hashtable no longer holds AggregateHashTable — from the table being taken earlier by on_finish/spill paths, double process/finish invocation, or pipeline driver bugs.

Common situations: Query finalization racing with spill-out; executor state-machine regressions; custom pipelines calling finish more than once.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/1e5eb09da7c9eec8. Report an issue: GitHub.

Appendix: source

Thrown at src/query/service/src/pipelines/processors/transforms/aggregator/transform_aggregate_final.rs:331

        } else {
            unreachable!("[TRANSFORM-AGGREGATOR] Invalid hash table state during spill check")
        }
        self.reset_hashtable(self.current_partition_depth);
        Ok(())
    }

    fn finish(
        &mut self,
        task_id: Option<u64>,
        spilled_depth: usize,
        tx: Sender<FinalAggregateTask>,
    ) -> Result<()> {
        if self.spilled_occurred {
            let (output_rows, hash_index_resizes) = match &self.hashtable {
                HashTable::AggregateHashTable(ht) => {
                    (ht.payload.len(), ht.hash_index_resize_count())
                }
                _ => unreachable!("[TRANSFORM-AGGREGATOR] Invalid hash table state before spill"),
            };
            self.spill_finish(spilled_depth, tx)?;
            if let Some(task_id) = task_id {
                self.statistics.log_task_finish_statistics(
                    task_id,
                    self._id,
                    spilled_depth,
                    output_rows,
                    hash_index_resizes,
                    true,
                );
            } else {
                self.statistics.reset();
            }

            self.spilled_occurred = false;
            let _ = mem::take(&mut self.hashtable);
            return Ok(());

View on GitHub (pinned to 288d84d76e)