tracel-ai/burn · error

q_matmul inputs are on different backends

Error message

q_matmul inputs are on different backends

What it means

This panic is the catch-all arm of the quantized-x-quantized q_matmul dispatch macro (q_matmul_qq_arms). The dispatcher holds each tensor tagged with its backend (DispatchTensorKind), and it only has compiled arms for both operands carrying the SAME backend. When lhs and rhs carry different backend tags the wildcard arm is reached and burn panics, because a backend-specific q_matmul kernel cannot operate across two different backends.

Source

Thrown at crates/burn-dispatch/src/ops/qtensor.rs:81

    }};
}

macro_rules! q_matmul_qq_arms {
    ($lhs:expr, $rhs:expr, $autodiff:expr; $([$Backend:ident, $cfg:meta]),*) => {{
        match ($lhs.kind, $rhs.kind) {
            $(
                #[cfg($cfg)]
                (DispatchTensorKind::$Backend(lhs), DispatchTensorKind::$Backend(rhs)) => {
                    type B = crate::backends::$Backend;
                    let output = B::q_matmul(
                        TensorPrimitive::QFloat(lhs.quantized()),
                        TensorPrimitive::QFloat(rhs.quantized()),
                    );
                    wrap_q_matmul_concrete!($Backend, output, $autodiff)
                }
            )*
            #[allow(unreachable_patterns)]
            _ => panic!("q_matmul inputs are on different backends"),
        }
    }};
}

macro_rules! q_matmul_fq_arms {
    ($lhs:expr, $rhs:expr, $autodiff:expr; $([$Backend:ident, $cfg:meta]),*) => {{
        match ($lhs.kind, $rhs.kind) {
            $(
                #[cfg($cfg)]
                (DispatchTensorKind::$Backend(lhs), DispatchTensorKind::$Backend(rhs)) => {
                    match $autodiff {
                        DispatchAutodiffContext::Disabled => {
                            type B = crate::backends::$Backend;
                            let output = B::q_matmul(
                                TensorPrimitive::Float(lhs.float()),
                                TensorPrimitive::QFloat(rhs.quantized()),
                            );
                            wrap_q_matmul_concrete!($Backend, output, DispatchAutodiffContext::Disabled)

View on GitHub (pinned to d16f7ba2ed)

Solutions

1. Ensure both quantized operands are created by (or moved to) the same backend before the matmul
2. Move one tensor to the other's backend/device using to_device / re-creation on the same backend
3. Audit generic code so both operands flow through the same Backend type parameter
4. Check that conversions (quantize/dequantize) did not swap the underlying backend of one operand

Example fix

// before
let a = QTensor::from_data(data_a, &wgpu_device); // wgpu backend
let b = QTensor::from_data(data_b, &tch_device);  // tch backend
let c = a.matmul(b); // panic: different backends
// after
let b = QTensor::from_data(data_b, &wgpu_device); // same backend/device
let c = a.matmul(b);

fn same_backend_qq(lhs: &DispatchTensor, rhs: &DispatchTensor) -> bool {
    core::mem::discriminant(&lhs.kind) == core::mem::discriminant(&rhs.kind)
}
assert!(same_backend_qq(&lhs, &rhs), "operands must share a backend before q_matmul");

fn is_qfloat(t: &DispatchTensor) -> bool {
    !matches!(t.kind, DispatchTensorKind::Autodiff(_)) && t.autodiff == DispatchAutodiffContext::Disabled
}
fn same_kind(a: &DispatchTensor, b: &DispatchTensor) -> bool {
    core::mem::discriminant(&a.kind) == core::mem::discriminant(&b.kind)
}

When it happens

Trigger: Calling q_matmul (or Tensor::matmul on quantized tensors) where the two DispatchTensors have different DispatchTensorKind variants — e.g. a tensor created on the cubecl-wgpu backend multiplied against one on the tch/candle backend, or one operand silently falling through to a different backend via a conversion.

Common situations: Mixing tensors created by different Backend types in generic code; moving one tensor to a different device/backend and forgetting the other; combining a quantized checkpoint loaded under one backend with a runtime tensor under another; generic functions parameterized over two different backend type parameters.

Related errors

• Quantization scheme is not valid for dtype {other:?}
• Can't store native sub-byte values
• {other:?} doesn't support native packing
• Tensor is on the wrong backend (expected {backend}).
• Autodiff float tensor is on the wrong backend (expected {bac

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/0ba6cd60c611f309. Report an issue: GitHub.