tracel-ai/burn · error

matmul: matrix size overflow: {a} * {b}

Error message

matmul: matrix size overflow: {a} * {b}

What it means

checked_size multiplies two matrix dimension sizes (e.g. M*N, K or batch dims) to size the GEMM output buffer. It uses checked_mul and panics when the product overflows usize, i.e. the requested matmul output is too large to address on this platform (practically only on 32-bit targets or with astronomically large dimensions). Called from matmul_batched_gemm, matmul_batched_i32 and matmul_batched_i64.

Source

Thrown at crates/burn-flex/src/ops/matmul.rs:53

    fn one() -> Self {
        1.0
    }
}

impl GemmScalar for f16 {
    fn zero() -> Self {
        f16::from_f32(0.0)
    }
    fn one() -> Self {
        f16::from_f32(1.0)
    }
}

/// Checked multiplication for matrix sizes, panics on overflow.
#[inline]
fn checked_size(a: usize, b: usize) -> usize {
    a.checked_mul(b)
        .unwrap_or_else(|| panic!("matmul: matrix size overflow: {a} * {b}"))
}

/// Threshold for enabling parallelism (M*N*K operations).
/// 192^3 = ~7M ops - balance between 128x128 (no parallel) and 256x256 (parallel)
const PARALLEL_THRESHOLD: usize = 192 * 192 * 192;

/// Threshold for batch-level parallelism (total ops across all batches).
/// Use batch parallelism when individual matrices are small but total work is large.
#[cfg(feature = "rayon")]
const BATCH_PARALLEL_THRESHOLD: usize = 128 * 128 * 128; // ~2M ops total

/// Get parallelism setting based on matrix size.
fn get_parallelism(m: usize, n: usize, k: usize) -> gemm::Parallelism {
    let ops = m.saturating_mul(n).saturating_mul(k);
    if ops >= PARALLEL_THRESHOLD {
        #[cfg(feature = "rayon")]
        {
            gemm::Parallelism::Rayon(0) // 0 = use all available threads

View on GitHub (pinned to d16f7ba2ed)

Solutions

1. Reduce tensor/batch dimensions; the requested output simply cannot fit in memory anyway.
2. Chunk the matmul into smaller batch slices and process sequentially.
3. On 32-bit/WASM targets, switch to 64-bit addressing or keep tensors under the 4GB element-count bound.
4. Fix upstream broadcasting/expand logic that multiplied batch dims unexpectedly.

Example fix

// before: one giant batched matmul
let out = a.matmul(b); // a shape [1_000_000_000, 4096, 4096]
// after
for chunk in a_chunks.iter() {
    let out_part = chunk.matmul(b); // bounded chunk sizes
}

let (m, n) = (a.shape()[a.rank() - 2], b.shape()[b.rank() - 1]);
assert!(m.checked_mul(n).is_some(), "matmul output size {}*{} overflows usize", m, n);

// panic-based; cannot be caught in-process. Validate before calling:
fn fits_usize(dims: &[usize]) -> bool { dims.iter().try_fold(1usize, |acc, &d| acc.checked_mul(d)).is_some() }

When it happens

Trigger: Calling matmul (Tensor::matmul) with shapes whose flattened output element count exceeds usize::MAX — e.g. huge batch dimensions times M*N on a 32-bit platform, or pathological shapes like [usize-large, k] @ [k, usize-large].

Common situations: 32-bit embedded/WASM targets with moderately large tensors; a broadcasting bug upstream inflating batch dimensions exponentially; accidental use of byte counts instead of element counts when reshaping.

Related errors

• matmul: unsupported dtype {:?}
• int_matmul: unsupported dtype {:?}
• capture tensor operations must run inside CaptureDevice::cap
• capture tensor {} has no initialized value
• seeding is not supported during graph capture

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/5c57577d875c6b63. Report an issue: GitHub.