{"record":{"id":"15c686b33f841548","repo":"tursodatabase/turso","slug":"memory-allocation-failed-here","errorCode":null,"errorMessage":"Memory allocation failed here","messagePattern":"Memory allocation failed here","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"core/vdbe/sorter.rs","lineNumber":966,"sourceCode":"\nimpl ArenaSortableRecord {\n    /// Full key comparison; only reached when the normalized keys tie.\n    fn full_cmp(&self, other: &Self) -> Ordering {\n        let self_values = self.key_values();\n        let other_values = other.key_values();\n        // SAFETY: index_key_info and comparators point to Sorter-owned data that outlives all records.\n        let index_key_info = unsafe { self.index_key_info.as_ref() };\n        let comparators = unsafe { self.comparators.as_ref() };\n\n        for (i, ((&self_val, &other_val), key_info)) in self_values\n            .iter()\n            .zip(other_values.iter())\n            .zip(index_key_info.iter())\n            .enumerate()\n        {\n            let cmp = if let Some(Some(comparator)) = comparators.get(i) {\n                let base =\n                    comparator(&self_val, &other_val).expect(\"Memory allocation failed here\");\n                cmp_with_sort(base, &self_val, &other_val, key_info)\n            } else {\n                cmp_in_column(&self_val, &other_val, key_info)\n            };\n            if cmp != Ordering::Equal {\n                return cmp;\n            }\n        }\n\n        Ordering::Equal\n    }\n}\n\nimpl Ord for ArenaSortableRecord {\n    #[inline]\n    fn cmp(&self, other: &Self) -> Ordering {\n        match self.norm_key.cmp(&other.norm_key) {\n            Ordering::Equal if self.norm_decisive && other.norm_decisive => Ordering::Equal,","sourceCodeStart":948,"sourceCodeEnd":984,"githubUrl":"https://github.com/tursodatabase/turso/blob/492c4a71cd7c2649e7df83da1471b74f4b1c7aa9/core/vdbe/sorter.rs#L948-L984","documentation":"The query sorter compares records using per-column comparators (SortComparator = Arc<dyn Fn(&ValueRef, &ValueRef) -> Result<Ordering>>, core/vdbe/sorter.rs:29) installed for custom collations or custom type ordering. Those comparators allocate - e.g. NumericLt converts each ValueRef to an owned Value (a.to_owned()?) to build a bigdecimal in core/vdbe/execute.rs:244-246 - so they can return Err on allocation failure. The sorter unwraps that with expect(\"Memory allocation failed here\") (core/vdbe/sorter.rs:959): the process ran out of memory while comparing sort keys during ORDER BY.","triggerScenarios":"A large ORDER BY (or any sorter-consuming plan, e.g. GROUP BY or index building) where an ORDER BY term has a comparator installed (custom collation via make_collation_comparator, or a custom-type comparator like NumericLt from SortComparatorType) and per-comparison Value allocations exhaust memory mid-sort.","commonSituations":"Sorting wide TEXT/BLOB columns with decimal-style custom comparisons, containers with cgroup memory caps, 32-bit builds, or several memory-heavy queries running concurrently until the allocator fails.","solutions":["Cut the sort working set: add LIMIT, filter rows before ORDER BY, or sort fewer/narrower columns.","Raise the memory ceiling (container/cgroup limit, host RAM) so per-comparison allocations succeed.","Add an index that satisfies the ORDER BY so the sorter is bypassed; verify with EXPLAIN that no sorter opcode remains.","If it reproduces with modest data, report it with RUST_BACKTRACE=1 - an OOM surfaced as a panic instead of LimsoError::OutOfMemory is arguably a bug in this code path."],"exampleFix":"-- before: full-table custom-collation sort\nSELECT * FROM huge ORDER BY decimal_col;\n\n-- after: bound the result and let an index provide order\nCREATE INDEX ix_dec ON huge(decimal_col);\nSELECT * FROM huge ORDER BY decimal_col LIMIT 100;","handlingStrategy":"validation","validationCode":"-- Run before executing a large sorted query: if EXPLAIN shows a sorter\n-- (SorterOpen/SorterInsert) and your ORDER BY column has a custom collation\n-- or custom type ordering, the comparator allocates per comparison.\nEXPLAIN QUERY PLAN SELECT * FROM huge ORDER BY decimal_col;\n-- Also check available memory before running: it must comfortably exceed\n-- the sorted dataset size, not just the row count.","typeGuard":null,"tryCatchPattern":"Python binding: wrap execute in try/except RuntimeError and re-raise as a memory-limit error so callers can shed load; treat it as unrecoverable for the query, not retryable.","preventionTips":["EXPLAIN every large ORDER BY; if a sorter is used, add LIMIT or an ordering index.","Set container/cgroup memory limits well above the largest expected sort working set.","Avoid custom collations on ORDER BY columns of unbounded tables.","Monitor process RSS during sort-heavy workloads to catch pressure before the allocator fails."],"tags":["rust","panic","out-of-memory","sorter","order-by","comparator","collation","turso"],"backgroundTag":"out-of-memory","analyzedSha":"492c4a71cd7c2649e7df83da1471b74f4b1c7aa9","analyzedAt":"2026-08-20T07:02:18.389Z","contentChangedAt":"2026-08-20T07:02:18.389Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}