{"record":{"id":"8f60d5d5f5a69cf7","repo":"databendlabs/databend","slug":"transform-aggregator-invalid-hash-table-state-du","errorCode":null,"errorMessage":"[TRANSFORM-AGGREGATOR] Invalid hash table state during spill check","messagePattern":"\\[TRANSFORM-AGGREGATOR\\] Invalid hash table state during spill check","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/query/service/src/pipelines/processors/transforms/aggregator/transform_aggregate_final.rs","lineNumber":314,"sourceCode":"                        check_interrupt()?;\n                        self.handle_meta(AggregateMeta::from(item), need_check_spill)?;\n                    }\n                }\n            },\n        }\n        Ok(())\n    }\n\n    fn spill_out(&mut self) -> Result<()> {\n        self.spilled_occurred = true;\n        if let HashTable::AggregateHashTable(v) = mem::take(&mut self.hashtable) {\n            for (bucket, payload) in v.payload.into_non_empty_bucket_payloads() {\n                check_interrupt()?;\n                let data_block = payload.aggregate_flush_all()?.consume_convert_to_full();\n                self.spiller.spill(bucket, data_block)?;\n            }\n        } else {\n            unreachable!(\"[TRANSFORM-AGGREGATOR] Invalid hash table state during spill check\")\n        }\n        self.reset_hashtable(self.current_partition_depth);\n        Ok(())\n    }\n\n    fn finish(\n        &mut self,\n        task_id: Option<u64>,\n        spilled_depth: usize,\n        tx: Sender<FinalAggregateTask>,\n    ) -> Result<()> {\n        if self.spilled_occurred {\n            let (output_rows, hash_index_resizes) = match &self.hashtable {\n                HashTable::AggregateHashTable(ht) => {\n                    (ht.payload.len(), ht.hash_index_resize_count())\n                }\n                _ => unreachable!(\"[TRANSFORM-AGGREGATOR] Invalid hash table state before spill\"),\n            };","sourceCodeStart":296,"sourceCodeEnd":332,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/service/src/pipelines/processors/transforms/aggregator/transform_aggregate_final.rs#L296-L332","documentation":"TransformFinalAggregate::spill_out spills per-bucket payloads and expects self.hashtable to be HashTable::AggregateHashTable; if it has been moved out (HashTable::MovedOut) or is otherwise in an invalid state, unreachable! aborts. It guards the invariant that a spill check can only run while the final aggregation hash table is still owned by the transform.","triggerScenarios":"check_spill -> spill_out runs after the hash table was already taken (e.g. via std::mem::take in on_finish or MovedOut marking), so the else branch fires — typically from double-finish calls, pipeline event ordering bugs, or interrupted/retried pipeline execution.","commonSituations":"Query cancellation racing with spill; executor bugs invoking finish and spill paths out of order; regressions after refactoring the HashTable enum state machine.","solutions":["Upgrade to a version with the transform state-machine fix and retry the query","Check for duplicate finish/spill invocation in any custom pipeline wiring","File a bug with the stack trace and query profile if reproducible"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":"fn hashtable_present(ht: &HashTable) -> Option<&AggregateHashTable> {\n    match ht {\n        HashTable::AggregateHashTable(t) => Some(t),\n        HashTable::MovedOut => None,\n    }\n}","tryCatchPattern":"match res {\n    Err(e) if e.message().contains(\"Invalid hash table state during spill check\") => {\n        // retry query; reduce memory pressure or disable spill path; report if reproducible\n    }\n    ...\n}","preventionTips":["Avoid custom pipeline wiring that can call finish before spill checks","Keep executor lifecycle tests for spill + finish interleavings","Upgrade promptly; such state-machine bugs are fixed in patch releases","Watch memory/spill settings so spill paths execute under normal conditions"],"tags":["internal","aggregation","spill","unreachable"],"backgroundTag":"internal-invariant-violation","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}