huggingface/candle · error

not implemented yet, self.device: {:?}, device: {:?}

Error message

not implemented yet, self.device: {:?}, device: {:?}

What it means

Tensor::to_device (and related device-transfer paths) only implements transfers for known storage/device combinations: Cpu->Cuda, Cuda->Cpu, Cpu->Cpu, and Cuda->Cuda. Any other combination, such as transfers involving Metal storage or an unexpected Storage variant, hits the catch-all arm and bails with this message, reporting the tensor's current device and the requested target device.

Source

Thrown at candle-core/src/tensor.rs:2416

            Ok(self.clone())
        } else {
            let storage = match (&*self.storage(), device) {
                (Storage::Cpu(storage), Device::Cuda(cuda)) => {
                    Storage::Cuda(cuda.storage_from_cpu_storage(storage)?)
                }
                (Storage::Cpu(storage), Device::Metal(metal)) => {
                    Storage::Metal(metal.storage_from_cpu_storage(storage)?)
                }
                (Storage::Cuda(storage), Device::Cpu) => Storage::Cpu(storage.to_cpu_storage()?),
                (Storage::Metal(storage), Device::Cpu) => Storage::Cpu(storage.to_cpu_storage()?),
                (Storage::Cuda(storage), Device::Cuda(cuda)) => {
                    // can't clone storage if it's the same device because of the underlying device ptr
                    let dst_storage = storage.transfer_to_device(cuda)?;
                    Storage::Cuda(dst_storage)
                }
                (Storage::Cpu(storage), Device::Cpu) => Storage::Cpu(storage.clone()),
                _ => {
                    bail!(
                        "not implemented yet, self.device: {:?}, device: {:?}",
                        self.device(),
                        device
                    )
                }
            };
            let op = BackpropOp::new1(self, Op::ToDevice);
            let tensor_ = Tensor_ {
                id: TensorId::new(),
                storage: Arc::new(RwLock::new(storage)),
                layout: self.layout.clone(),
                op,
                is_variable: false,
                dtype: self.dtype,
                device: device.clone(),
            };
            Ok(Tensor(Arc::new(tensor_)))
        }

View on GitHub (pinned to d5fee525bf)

Solutions

1. Check which device pair is unsupported and upgrade candle to a version that implements transfer_to_device for that backend
2. Transfer via an intermediate device explicitly (e.g. Metal -> Cpu -> Cuda) only if both individual paths are implemented
3. Avoid the unsupported transfer: keep the tensor on its original device and pass the target device at model load/creation time
4. Use backend-appropriate APIs (e.g. Metal tensor methods) instead of generic to_device

Example fix

// before
let t = t.to_device(&Device::Cpu)?; // Metal tensor: bails 'not implemented yet'
// after
// compile with the cuda feature or verify self.device() supports the transfer first
let t = if t.device().is_cuda() || t.device().is_cpu() { t.to_device(&Device::Cpu)? } else { t };

fn can_transfer(dev: &candle_core::Device, target: &candle_core::Device) -> bool {
    use candle_core::Device::*;
    matches!((dev, target), (Cpu, Cpu) | (Cpu, Cuda(_)) | (Cuda(_), Cpu) | (Cuda(_), Cuda(_)))
}
// call: assert!(can_transfer(&t.device(), &target_device));

match t.to_device(&target) {
    Ok(v) => v,
    Err(e) if e.to_string().contains("not implemented yet") => fallback_cpu_path(),
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling tensor.to_device(Device::Cpu) on a Metal-backed tensor, or to_device between devices where the (Storage, Device) pattern match falls into the `_` arm (e.g. Storage::Cuda with Device::Metal, or a device variant lacking a transfer implementation).

Common situations: Running models on Metal or other backends and then calling .to_device() assuming CPU/GPU transfer support exists; mixing backends after loading a model saved from a different device setup; code written for CUDA builds being reused on Metal builds.

Related errors

• {} is a dummy type and cannot be constructed
• {} is a dummy type and cannot be converted
• {} is a dummy type and cannot be converted to scalar
• {} is a dummy type and does not support storage
• {} is a dummy type and does not support operations

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/0583b4423b3144ea. Report an issue: GitHub.