What causes "Metal device does not yet support F8E4M3." in huggingface/candle?

Cast to a supported dtype before sorting: tensor.to_dtype(DType::F32)? then arg_sort

Metal device does not yet support F8E4M3.

Error message

Metal device does not yet support F8E4M3.

What it means

The Metal argsort kernel set has no F8E4M3 variant, so metal_fwd explicitly bails when asked to argsort an F8E4M3 tensor. Other exotic float types (F6E2M3, F6E3M2, F4, F8E8M0) raise the related UnsupportedDTypeForOp error instead.

Source

Thrown at candle-core/src/sort.rs:193

        storage: &crate::MetalStorage,
        layout: &crate::Layout,
    ) -> Result<(crate::MetalStorage, crate::Shape)> {
        use crate::backend::BackendStorage;
        use crate::DType;

        let name = {
            if self.asc {
                match storage.dtype() {
                    DType::BF16 => "asort_asc_bf16",
                    DType::F16 => "asort_asc_f16",
                    DType::F32 => "asort_asc_f32",
                    DType::F64 => "asort_asc_f64",
                    DType::U8 => "asort_asc_u8",
                    DType::U32 => "asort_asc_u32",
                    DType::I16 => "asort_asc_i16",
                    DType::I32 => "asort_asc_i32",
                    DType::I64 => "asort_asc_i64",
                    DType::F8E4M3 => crate::bail!("Metal device does not yet support F8E4M3."),
                    DType::F6E2M3 | DType::F6E3M2 | DType::F4 | DType::F8E8M0 => {
                        return Err(
                            crate::Error::UnsupportedDTypeForOp(storage.dtype(), "argsort").bt(),
                        )
                    }
                }
            } else {
                match storage.dtype() {
                    DType::BF16 => "asort_desc_bf16",
                    DType::F16 => "asort_desc_f16",
                    DType::F32 => "asort_desc_f32",
                    DType::F64 => "asort_desc_f64",
                    DType::U8 => "asort_desc_u8",
                    DType::U32 => "asort_desc_u32",
                    DType::I16 => "asort_desc_i16",
                    DType::I32 => "asort_desc_i32",
                    DType::I64 => "asort_desc_i64",
                    DType::F8E4M3 => crate::bail!("Metal device does not yet support F8E4M3."),

View on GitHub (pinned to d5fee525bf)

Solutions

Cast to a supported dtype before sorting: tensor.to_dtype(DType::F32)? then arg_sort
Perform the argsort on CPU (move the tensor with .to_device(&Device::Cpu)?)
Avoid fp8 tensors on the Metal backend for sort-like ops; keep them only where supported kernels exist

Example fix

// before
let idx = x.arg_sort(Ascending)?; // x: F8E4M3 on Metal
// after
let idx = x.to_dtype(DType::F32)?.arg_sort(Ascending)?;

Defensive patterns

Strategy: fallback

Validate before calling

if x.dtype() == candle::DType::F8E4M3 && x.device().is_metal() {
    x = x.to_dtype(candle::DType::F32)?;
}
let idx = x.arg_sort(Ascending)?;

Try / catch

let idx = x.arg_sort(Ascending).or_else(|_| {
    if x.device().is_metal() { x.to_dtype(candle::DType::F32)?.arg_sort(Ascending) } else { unreachable!() }
})?;

Prevention

Avoid fp8 (F8E4M3) tensors on the Metal backend for sort/argsort
Cast to F32/F16 before GPU ops with limited dtype kernel coverage on Metal
Check candle's Metal kernel list before porting fp8 CUDA pipelines to macOS

When it happens

Trigger: Calling arg_sort (or sort that routes through argsort) on a tensor whose dtype is F8E4M3 while running on the Metal backend (Apple Silicon GPU).

Common situations: Casting activations/weights to fp8 (F8E4M3) for inference on macOS/Metal and then sorting logits or values; porting CUDA fp8 pipelines to Metal.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/30013b74eecb2eb9. Report an issue: GitHub.