tracel-ai/burn · error

todo!("Local transfer of {dtype:?} tensors is not supported

Error message

todo!("Local transfer of {dtype:?} tensors is not supported yet")

What it means

In burn-router's interpreter, `get_tensor` moves a primitive to the local device by matching dtype to Float/Int/Bool handles. Any other dtype (e.g. quantized dtypes like QInt8 or exotic numeric types) has no local transfer path, so the code hits `todo!` and panics with 'Local transfer of {dtype:?} tensors is not supported yet'.

Source

Thrown at crates/burn-router/src/interpreter.rs:98

    }

    /// Take the typed backend primitive for `tensor`, dispatching on its dtype.
    ///
    /// Unlike [`get_tensor_handle`](Self::get_tensor_handle) (which returns the opaque
    /// `B::Handle`), this returns the concrete float/int/bool primitive so the caller can hand
    /// it to `B::*_to_device`. Used by the same-host transfer path, which moves a tensor between
    /// two interpreters living in the same server process without a host round-trip.
    pub fn get_tensor(&mut self, tensor: &TensorIr) -> HandleKind<B> {
        let handles = &mut self.context.handles;
        let dtype = tensor.dtype;
        if dtype.is_float() {
            HandleKind::Float(handles.get_float_tensor::<B>(tensor))
        } else if dtype.is_int() {
            HandleKind::Int(handles.get_int_tensor::<B>(tensor))
        } else if dtype.is_bool() {
            HandleKind::Bool(handles.get_bool_tensor::<B>(tensor))
        } else {
            todo!("Local transfer of {dtype:?} tensors is not supported yet");
        }
    }

    /// Move a primitive produced on another interpreter's device onto this interpreter's device
    /// and register it under `id`.
    ///
    /// The counterpart of [`get_tensor`](Self::get_tensor): the source interpreter hands over its
    /// primitive, and the destination calls `B::*_to_device` onto its own device. When both
    /// interpreters share the same device, the backend's `to_device` is a cheap no-op.
    pub fn register_tensor_to_device(&mut self, id: TensorId, tensor: HandleKind<B>) {
        let ctx = &mut self.context;
        match tensor {
            HandleKind::Float(tensor) => {
                let tensor = B::float_to_device(tensor, &self.device);
                ctx.handles.register_float_tensor::<B>(&id, tensor);
            }
            HandleKind::Int(tensor) => {
                let tensor = B::int_to_device(tensor, &self.device);

View on GitHub (pinned to d16f7ba2ed)

Solutions

1. Enable the quantization feature/paths so quantized tensors are handled by a dedicated code path instead of local transfer.
2. Keep quantized tensors on the device where they were produced; dequantize to float before transferring.
3. Use float (f32) tensors for the operation and quantize afterwards locally.
4. Update burn to a version where quantized local transfer is implemented.

Example fix

// before
let q = model_quantized.output().to_device(&other_device); // panics
// after
let f = model_quantized.output().dequantize();
let f = f.to_device(&other_device);

fn transferable(dtype: DType) -> bool {
    dtype.is_float() || dtype.is_int() || dtype.is_bool()
}
assert!(transferable(t.dtype()), "dequantize before device transfer");

fn is_quantized_dtype(d: DType) -> bool { !d.is_float() && !d.is_int() && !d.is_bool() }

When it happens

Trigger: Executing a remote/multi-router operation where a tensor with a non-float/int/bool dtype (typically a quantized dtype) must be transferred to the local router device — e.g. fetching or using a quantized tensor result through the router without a quantization feature enabled.

Common situations: Running quantized inference across router-managed backends/devices; moving a quantized checkpoint tensor between GPUs/devices; a dtype added upstream but not yet wired into the router's handle transfer.

Related errors

• todo!("Local transfer of quantized tensors is not supported
• todo!("Quantization not supported yet")
• Can't differentiate deform conv 2d backward.
• Quantization scheme is not valid for dtype {other:?}
• Can't store native sub-byte values

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/df82db8ef1c1e562. Report an issue: GitHub.