{"record":{"id":"0d8cfe46f095ac18","repo":"databendlabs/databend","slug":"transform-aggregator-invalid-hash-table-state-du-0d8cfe","errorCode":null,"errorMessage":"[TRANSFORM-AGGREGATOR] Invalid hash table state during spill check","messagePattern":"\\[TRANSFORM-AGGREGATOR\\] Invalid hash table state during spill check","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/query/service/src/pipelines/processors/transforms/aggregator/transform_aggregate_partial.rs","lineNumber":223,"sourceCode":"            }\n        }\n    }\n\n    fn spill_out(&mut self) -> Result<()> {\n        if let HashTable::AggregateHashTable(v) = std::mem::take(&mut self.hash_table) {\n            let config = v.config.clone();\n\n            self.spillers.spill(v.payload, self.is_row_shuffle)?;\n\n            let arena = Arc::new(Bump::new());\n            self.hash_table = HashTable::AggregateHashTable(AggregateHashTable::new(\n                self.params.group_data_types.clone(),\n                self.params.aggregate_functions.clone(),\n                config,\n                arena,\n            ));\n        } else {\n            unreachable!(\"[TRANSFORM-AGGREGATOR] Invalid hash table state during spill check\")\n        }\n        Ok(())\n    }\n}\n\nimpl AccumulatingTransform for TransformPartialAggregate {\n    const NAME: &'static str = \"TransformPartialAggregate\";\n\n    fn transform(&mut self, block: DataBlock) -> Result<Vec<DataBlock>> {\n        self.execute_one_block(block)?;\n\n        if self.settings.check_spill() {\n            self.spill_out()?;\n        }\n\n        Ok(vec![])\n    }\n","sourceCodeStart":205,"sourceCodeEnd":241,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/service/src/pipelines/processors/transforms/aggregator/transform_aggregate_partial.rs#L205-L241","documentation":"TransformPartialAggregate::spill_out rebuilds/reads the hash table for spilling and expects HashTable::AggregateHashTable; any other state (MovedOut) hits unreachable!(\"Invalid hash table state during spill check\"). It asserts spills only occur while the partial aggregation table is still owned.","triggerScenarios":"check_spill triggers spill_out after the table was taken/moved out — from finish running before spill checks complete, pipeline driver ordering bugs, or cancellation racing with spill.","commonSituations":"Large aggregation spilling to disk combined with early finish; executor state-machine regressions; duplicated transform invocations in custom pipelines.","solutions":["Update to a fixed version and rerun the query","Ensure the pipeline invokes spill checks only before on_finish","File a bug with the stack trace and memory/spill settings if reproducible"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":"fn can_spill(ht: &HashTable) -> bool {\n    matches!(ht, HashTable::AggregateHashTable(_))\n}","tryCatchPattern":"match res {\n    Err(e) if e.message().contains(\"Invalid hash table state during spill check\") => {\n        // retry; tune memory limits so spill happens mid-execution, not at finish\n    }\n    ...\n}","preventionTips":["Verify spill checks only run during the transform phase, before on_finish","Include spill interleavings in executor tests","Upgrade promptly when hitting this in large aggregations","Avoid duplicate driver invocations of check_spill after finish"],"tags":["internal","aggregation","spill","unreachable"],"backgroundTag":"internal-invariant-violation","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}