{"record":{"id":"5e578dfd967bcd18","repo":"huggingface/candle","slug":"rope-thd-is-not-implemented-for-dtype","errorCode":null,"errorMessage":"rope_thd is not implemented for {dtype:?}","messagePattern":"rope_thd is not implemented for (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/rotary_emb.rs","lineNumber":792,"sourceCode":"    ) -> Result<(candle::MetalStorage, Shape)> {\n        use candle::backend::BackendStorage;\n        let device = src.device();\n        let encoder = device.command_encoder()?;\n        encoder.set_label(\"rope_thd\");\n        let kernels = device.kernels();\n        if cos.dtype() != src.dtype() || sin.dtype() != src.dtype() {\n            candle::bail!(\n                \"dtype mismatch in rope {:?} {:?} {:?}\",\n                src.dtype(),\n                cos.dtype(),\n                sin.dtype()\n            )\n        }\n        let name = match src.dtype() {\n            candle::DType::F32 => \"rope_thd_f32\",\n            candle::DType::F16 => \"rope_thd_f16\",\n            candle::DType::BF16 => \"rope_thd_bf16\",\n            dtype => candle::bail!(\"rope_thd is not implemented for {dtype:?}\"),\n        };\n        let (b, t, h, d) = l_src.shape().dims4()?;\n        let stride_b = if l_cos.dims().len() == 3 && l_sin.dims().len() == 3 {\n            h * t * d\n        } else {\n            0usize\n        };\n        let el = b * h * t * d;\n        let output = device\n            .new_buffer_builder()\n            .with_size_for(el, src.dtype())\n            .with_label(\"rope_thd\")\n            .build()?;\n        candle_metal_kernels::call_rope_thd(\n            device.metal_device(),\n            &encoder,\n            kernels,\n            name,","sourceCodeStart":774,"sourceCodeEnd":810,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/rotary_emb.rs#L774-L810","documentation":"The Metal rope_thd kernel is only compiled for F32, F16 and BF16; the dtype-to-kernel-name match in metal_fwd bails for any other dtype (e.g. F64, which has no metal kernel here, or integer types). This is a Metal-specific capability limitation.","triggerScenarios":"Calling rope_thd on Metal with an F64 tensor (F64 is supported on CPU/CUDA but has no metal kernel), or with integer/byte tensors.","commonSituations":"Code that uses F64 generically on CPU being moved to an Apple GPU device; test code defaulting to F64; passing position index tensors (I64) by mistake.","solutions":["Cast the input to F32 (or F16/BF16) before rope on Metal.","Avoid F64 on GPU paths entirely; keep F64 math on CPU and cast results back.","Use xs.to_dtype(DType::F32)? at model boundaries when targeting Metal.","Check device placement: an F64 tensor on a Metal device will always fail here."],"exampleFix":"// before\nlet xs = Tensor::randn(0f64, 1f64, shape, &metal_dev)?; // f64 on metal\nlet x = rope.forward(&xs, &cos, &sin)?;\n// after\nlet xs = xs.to_dtype(DType::F32)?;\nlet cos = cos.to_dtype(DType::F32)?;\nlet sin = sin.to_dtype(DType::F32)?;\nlet x = rope.forward(&xs, &cos, &sin)?;","handlingStrategy":"validation","validationCode":"if !matches!(xs.dtype(), candle::DType::F32 | candle::DType::F16 | candle::DType::BF16) {\n    let xs = xs.to_dtype(candle::DType::F32)?;\n}","typeGuard":"fn metal_supported_float(d: candle::DType) -> bool {\n    matches!(d, candle::DType::F32 | candle::DType::F16 | candle::DType::BF16)\n}","tryCatchPattern":"let out = match rope.forward(&xs, &cos, &sin) {\n    Ok(o) => o,\n    Err(e) if e.to_string().contains(\"is not implemented for\") => {\n        let xs = xs.to_dtype(candle::DType::F32)?;\n        rope.forward(&xs, &cos.to_dtype(candle::DType::F32)?, &sin.to_dtype(candle::DType::F32)?)?\n    }\n    Err(e) => return Err(e),\n};","preventionTips":["Never use F64 tensors on Metal devices; restrict F64 to CPU-only code paths","Cast model inputs to F32/F16/BF16 at the device boundary","Check device placement when a tensor is created (F64 defaults are easy to create on any device)","Pin test tensors to the same dtype used in production inference"],"tags":["candle","rope","metal","dtype","unsupported-kernel"],"backgroundTag":"kernel-not-implemented-for-dtype","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}