{"record":{"id":"374dc47cd21f039f","repo":"quickwit-oss/quickwit","slug":"more-memory-released-than-allocated-should-never","errorCode":null,"errorMessage":"More memory released than allocated, should never happen.","messagePattern":"More memory released than allocated, should never happen\\.","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"quickwit/quickwit-search/src/search_permit_provider.rs","lineNumber":384,"sourceCode":"                self.assign_available_permits();\n            }\n            SearchPermitMessage::FreeWarmupSlot => {\n                self.num_warmup_slots_available += 1;\n                self.assign_available_permits();\n            }\n            SearchPermitMessage::Drop {\n                memory_size,\n                warmup_slot_freed,\n            } => {\n                // total_job_cost is decremented synchronously in `SearchPermit::Drop`. This eases\n                // testing (no need to wait for the queue to drain to observe the cost was updated)\n                if !warmup_slot_freed {\n                    self.num_warmup_slots_available += 1;\n                }\n                self.total_memory_allocated = self\n                    .total_memory_allocated\n                    .checked_sub(memory_size)\n                    .expect(\"More memory released than allocated, should never happen.\");\n                self.assign_available_permits();\n            }\n        }\n    }\n\n    fn pop_next_request_if_serviceable(&mut self) -> Option<SingleSplitPermitRequest> {\n        if self.num_warmup_slots_available == 0 {\n            return None;\n        }\n        let available_memory = self\n            .total_memory_budget\n            .checked_sub(self.total_memory_allocated)?;\n        let mut peeked = self.permits_requests.peek_mut()?;\n\n        assert!(\n            !peeked.is_empty(),\n            \"unexpected empty permits_requests present in the search permit provider queue\"\n        );","sourceCodeStart":366,"sourceCodeEnd":402,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-search/src/search_permit_provider.rs#L366-L402","documentation":"This panic fires in the search permit provider's memory accounting when a release would drive total_memory_allocated below zero. It means the internal bookkeeping between allocation and release is broken: someone released memory that was never charged. The library treats this as an unrecoverable invariant violation and panics via expect.","triggerScenarios":"handle_message processes a memory release message whose memory_size was never added to total_memory_allocated, or the same allocation is released twice (e.g. duplicate drop/ack messages racing in the actor event loop).","commonSituations":"Actor message duplication or reordering bugs, custom sources or client code holding SearchPermits across actor restarts, or modifications to the permit provider that release warmup slots/memory out of order.","solutions":["Check for double-release of the same permit (SearchPermit dropped more than once or release message sent twice)","Verify the memory allocation path actually increments total_memory_allocated for every release path that can fire","Reproduce with debug logging of alloc/release pairs to find the unpaired release","If hit after code changes, review recent changes to warmup slot freeing and assign_available_permits ordering"],"exampleFix":"// before\nself.total_memory_allocated = self.total_memory_allocated.checked_sub(memory_size).expect(\"...\");\n// after\nif let Some(new_total) = self.total_memory_allocated.checked_sub(memory_size) {\n    self.total_memory_allocated = new_total;\n} else {\n    error!(memory_size, total = self.total_memory_allocated, \"unpaired memory release\");\n    return; // or panic with diagnostic context\n}","handlingStrategy":"validation","validationCode":"// Before releasing, assert the allocation was tracked:\nassert!(memory_size <= provider.total_memory_allocated, \"release exceeds allocated memory\");","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never release the same SearchPermit allocation twice","Keep alloc/release pairs within the provider actor to preserve ordering","Add debug assertions or logs pairing alloc and release events","Run the failpoint/actor tests after touching permit accounting"],"tags":["rust","memory-accounting","panic","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}