tursodatabase/turso · critical

Memory allocation failed here

Error message

Memory allocation failed here

What it means

The query sorter compares records using per-column comparators (SortComparator = Arc<dyn Fn(&ValueRef, &ValueRef) -> Result<Ordering>>, core/vdbe/sorter.rs:29) installed for custom collations or custom type ordering. Those comparators allocate - e.g. NumericLt converts each ValueRef to an owned Value (a.to_owned()?) to build a bigdecimal in core/vdbe/execute.rs:244-246 - so they can return Err on allocation failure. The sorter unwraps that with expect("Memory allocation failed here") (core/vdbe/sorter.rs:959): the process ran out of memory while comparing sort keys during ORDER BY.

Source

Thrown at core/vdbe/sorter.rs:966

impl ArenaSortableRecord {
    /// Full key comparison; only reached when the normalized keys tie.
    fn full_cmp(&self, other: &Self) -> Ordering {
        let self_values = self.key_values();
        let other_values = other.key_values();
        // SAFETY: index_key_info and comparators point to Sorter-owned data that outlives all records.
        let index_key_info = unsafe { self.index_key_info.as_ref() };
        let comparators = unsafe { self.comparators.as_ref() };

        for (i, ((&self_val, &other_val), key_info)) in self_values
            .iter()
            .zip(other_values.iter())
            .zip(index_key_info.iter())
            .enumerate()
        {
            let cmp = if let Some(Some(comparator)) = comparators.get(i) {
                let base =
                    comparator(&self_val, &other_val).expect("Memory allocation failed here");
                cmp_with_sort(base, &self_val, &other_val, key_info)
            } else {
                cmp_in_column(&self_val, &other_val, key_info)
            };
            if cmp != Ordering::Equal {
                return cmp;
            }
        }

        Ordering::Equal
    }
}

impl Ord for ArenaSortableRecord {
    #[inline]
    fn cmp(&self, other: &Self) -> Ordering {
        match self.norm_key.cmp(&other.norm_key) {
            Ordering::Equal if self.norm_decisive && other.norm_decisive => Ordering::Equal,

View on GitHub (pinned to 492c4a71cd)

Solutions

1. Cut the sort working set: add LIMIT, filter rows before ORDER BY, or sort fewer/narrower columns.
2. Raise the memory ceiling (container/cgroup limit, host RAM) so per-comparison allocations succeed.
3. Add an index that satisfies the ORDER BY so the sorter is bypassed; verify with EXPLAIN that no sorter opcode remains.
4. If it reproduces with modest data, report it with RUST_BACKTRACE=1 - an OOM surfaced as a panic instead of LimsoError::OutOfMemory is arguably a bug in this code path.

Example fix

-- before: full-table custom-collation sort
SELECT * FROM huge ORDER BY decimal_col;

-- after: bound the result and let an index provide order
CREATE INDEX ix_dec ON huge(decimal_col);
SELECT * FROM huge ORDER BY decimal_col LIMIT 100;

-- Run before executing a large sorted query: if EXPLAIN shows a sorter
-- (SorterOpen/SorterInsert) and your ORDER BY column has a custom collation
-- or custom type ordering, the comparator allocates per comparison.
EXPLAIN QUERY PLAN SELECT * FROM huge ORDER BY decimal_col;
-- Also check available memory before running: it must comfortably exceed
-- the sorted dataset size, not just the row count.

Python binding: wrap execute in try/except RuntimeError and re-raise as a memory-limit error so callers can shed load; treat it as unrecoverable for the query, not retryable.

When it happens

Trigger: A large ORDER BY (or any sorter-consuming plan, e.g. GROUP BY or index building) where an ORDER BY term has a comparator installed (custom collation via make_collation_comparator, or a custom-type comparator like NumericLt from SortComparatorType) and per-comparison Value allocations exhaust memory mid-sort.

Common situations: Sorting wide TEXT/BLOB columns with decimal-style custom comparisons, containers with cgroup memory caps, 32-bit builds, or several memory-heavy queries running concurrently until the allocator fails.

Related errors

• result should be set
• OpenRead on sorter cursor
• failed to generate random bytes
• Transaction dropped unexpectedly.
• Invalid drop behavior: {value}

AI-assisted analysis of tursodatabase/turso@492c4a71cd (2026-08-20). Data as JSON: /api/errors/15c686b33f841548. Report an issue: GitHub.