tracel-ai/burn · error

todo!("Local transfer of quantized tensors is not supported

Error message

todo!("Local transfer of quantized tensors is not supported yet")

What it means

In burn-router's interpreter, `register_tensor_to_device` moves handles to `self.device` per HandleKind (Float/Int/Bool). The `HandleKind::Quantized` variant has no device-move implementation, so registering a quantized tensor to another device panics via `todo!` with 'Local transfer of quantized tensors is not supported yet'.

Source

Thrown at crates/burn-router/src/interpreter.rs:124

    /// primitive, and the destination calls `B::*_to_device` onto its own device. When both
    /// interpreters share the same device, the backend's `to_device` is a cheap no-op.
    pub fn register_tensor_to_device(&mut self, id: TensorId, tensor: HandleKind<B>) {
        let ctx = &mut self.context;
        match tensor {
            HandleKind::Float(tensor) => {
                let tensor = B::float_to_device(tensor, &self.device);
                ctx.handles.register_float_tensor::<B>(&id, tensor);
            }
            HandleKind::Int(tensor) => {
                let tensor = B::int_to_device(tensor, &self.device);
                ctx.handles.register_int_tensor::<B>(&id, tensor);
            }
            HandleKind::Bool(tensor) => {
                let tensor = B::bool_to_device(tensor, &self.device);
                ctx.handles.register_bool_tensor::<B>(&id, tensor);
            }
            HandleKind::Quantized(_) => {
                todo!("Local transfer of quantized tensors is not supported yet");
            }
        }
    }

    /// Create a tensor with the given handle and shape.
    pub fn register_tensor<C: RouterClient>(
        &mut self,
        handle: B::Handle,
        shape: Shape,
        dtype: DType,
        client: C,
    ) -> RouterTensor<C> {
        let ctx = &mut self.context;
        let id = ctx.create_empty_handle();

        ctx.handles.register_handle(id, handle);

        RouterTensor::new(id, shape, dtype, client)

View on GitHub (pinned to d16f7ba2ed)

Solutions

1. Dequantize the tensor before moving devices, then re-quantize on the target device if needed.
2. Pin the quantized computation to a single device to avoid transfers.
3. Run on a single-device backend (no router) for quantized workloads.
4. Upgrade burn to a release that implements quantized device transfer.

Example fix

// before
quant_tensor.to_device(&device_b); // todo! panic
// after
let f = quant_tensor.dequantize().to_device(&device_b);
let q = f.quantize(&q_params, scheme);

fn can_move_to_device(h: &HandleKind) -> bool {
    !matches!(h, HandleKind::Quantized(_))
}
if !can_move_to_device(&handle) { /* dequantize first */ }

fn is_quantized_handle(h: &HandleKind) -> bool { matches!(h, HandleKind::Quantized(_)) }

When it happens

Trigger: Calling `.to_device()` (or any router operation that re-registers a tensor on a target device) on a quantized tensor whose handle kind is Quantized.

Common situations: Deploying quantized models across multiple devices with the router backend; running inference on a different GPU than where quantization happened; using a newer quantization API ahead of router support.

Related errors

• todo!("Local transfer of {dtype:?} tensors is not supported
• todo!("Quantization not supported yet")
• Can't differentiate deform conv 2d backward.
• Quantization scheme is not valid for dtype {other:?}
• Can't store native sub-byte values

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/d3eff36f3e4fc386. Report an issue: GitHub.