huggingface/candle · error

Metal contiguous to_dtype {left:?} {right:?} not implemented

Error message

Metal contiguous to_dtype {left:?} {right:?} not implemented

What it means

candle's Metal backend only implements a fixed table of dtype-pair casts for contiguous to_dtype operations; any (from,to) pair outside the table hits the catch-all bail arm. This means the requested dtype conversion has no Metal kernel available, so the library refuses rather than silently falling back. It is a backend-coverage limitation, not user data corruption.

Source

Thrown at candle-core/src/metal_backend/mod.rs:597

                (DType::I64, DType::F16) => "cast_i64_f16",
                (DType::I64, DType::F32) => "cast_i64_f32",
                (DType::I64, DType::U32) => "cast_i64_u32",
                (DType::I64, DType::U8) => "cast_i64_u8",

                (DType::F16, DType::BF16) => "cast_f16_bf16",
                (DType::F16, DType::F32) => "cast_f16_f32",
                (DType::F16, DType::I64) => "cast_f16_i64",
                (DType::F16, DType::U32) => "cast_f16_u32",
                (DType::F16, DType::U8) => "cast_f16_u8",

                (DType::BF16, DType::F16) => "cast_bf16_f16",
                (DType::BF16, DType::F32) => "cast_bf16_f32",
                (DType::BF16, DType::I64) => "cast_bf16_i64",
                (DType::BF16, DType::U32) => "cast_bf16_u32",
                (DType::BF16, DType::U8) => "cast_bf16_u8",

                (left, right) => {
                    crate::bail!("Metal contiguous to_dtype {left:?} {right:?} not implemented")
                }
            };
            candle_metal_kernels::call_cast_contiguous(
                &device.device,
                &encoder,
                &device.kernels,
                kernel_name,
                self.dtype.size_in_bytes(),
                el_count,
                src,
                &buffer,
            )
            .map_err(MetalError::from)?;
        } else {
            let kernel_name = match (self.dtype, dtype) {
                (DType::BF16, DType::F16) => "cast_bf16_f16_strided",
                (DType::BF16, DType::F32) => "cast_bf16_f32_strided",
                (DType::BF16, DType::I64) => "cast_bf16_i64_strided",

View on GitHub (pinned to d5fee525bf)

Solutions

  1. Cast on CPU first (tensor.to_device(&Device::Cpu)?.to_dtype(dtype)? then move back to Metal)
  2. Use a supported dtype pair from the cast table (F32/F16/BF16/U8/U32/I64 combos that are listed)
  3. Convert the tensor to F32 first, then to the target dtype, if both legs are supported
  4. Run the workload on the CUDA or CPU backend which has broader cast coverage

Example fix

// before
let t = tensor.to_device(&metal_dev)?.to_dtype(DType::I64)?;
// after
let t = tensor.to_dtype(DType::I64)?; // CPU-side cast, then .to_device(&metal_dev)
Defensive patterns

Strategy: validation

Validate before calling

fn metal_cast_supported(from: candle_core::DType, to: candle_core::DType) -> bool {
    use candle_core::DType::*;
    matches!(
        (from, to),
        (F32, F16) | (F32, BF16) | (F32, I64) | (F32, U32) | (F32, U8)
            | (F16, F32) | (BF16, F32) | (F16, BF16) | (BF16, F16)
        // extend with the pairs your candle version's table lists
    )
}
if !metal_cast_supported(t.dtype(), target) { t = t.to_device(&Device::Cpu)?.to_dtype(target)?.to_device(&metal_dev)?; }

Try / catch

match tensor.to_dtype(target) {
    Ok(t) => t,
    Err(e) if e.to_string().contains("to_dtype") && e.to_string().contains("not implemented") => {
        tensor.to_device(&Device::Cpu)?.to_dtype(target)?.to_device(&metal_dev)?
    }
    Err(e) => return Err(e.into()),
}

Prevention

When it happens

Trigger: Calling Tensor::to_dtype (or an op that casts internally) on a Metal device with a dtype pair not in the cast table, e.g. converting I64->F16, F32->I64 via the contiguous path, or any cast involving F64-style dtypes.

Common situations: Loading model weights stored in one dtype (e.g. i64 token ids) and casting to half precision for GPU inference; mixed-precision training pipelines converting between bf16 and integer types; dtype conversions that work on CPU but not on Metal.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/25a6d9f92ba14569. Report an issue: GitHub.