{"record":{"id":"6176887a43b7dee6","repo":"huggingface/candle","slug":"invalid-quantize-storage-locations-do-not-match","errorCode":null,"errorMessage":"Invalid quantize storage locations do not match","messagePattern":"Invalid quantize storage locations do not match","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/mod.rs","lineNumber":174,"sourceCode":"        }\n    }\n\n    fn size_in_bytes(&self) -> usize {\n        match self {\n            QStorage::Cpu(storage) => storage.storage_size_in_bytes(),\n            QStorage::Metal(storage) => storage.storage_size_in_bytes(),\n            QStorage::Cuda(storage) => storage.storage_size_in_bytes(),\n        }\n    }\n\n    fn quantize(&mut self, src: &Storage) -> Result<()> {\n        match (self, src) {\n            (QStorage::Cpu(storage), Storage::Cpu(src)) => {\n                storage.from_float(src.as_slice::<f32>()?);\n            }\n            (QStorage::Metal(storage), Storage::Metal(src)) => storage.quantize(src)?,\n            (QStorage::Cuda(storage), Storage::Cuda(src)) => storage.quantize(src)?,\n            _ => crate::bail!(\"Invalid quantize storage locations do not match\"),\n        }\n        Ok(())\n    }\n\n    fn quantize_imatrix(\n        &mut self,\n        src: &Storage,\n        imatrix_weights: &[f32],\n        n_per_row: usize,\n    ) -> Result<()> {\n        match (self, src) {\n            (QStorage::Cpu(storage), Storage::Cpu(src)) => {\n                storage.from_float_imatrix(src.as_slice::<f32>()?, imatrix_weights, n_per_row);\n            }\n            (QStorage::Metal(storage), Storage::Metal(src)) => {\n                storage.quantize_imatrix(src, imatrix_weights, n_per_row)?\n            }\n            (QStorage::Cuda(storage), Storage::Cuda(src)) => {","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/mod.rs#L156-L192","documentation":"QuantizedStorage::quantize requires the quantized destination storage and the f32 source storage to live on the same device. Any (QStorage, Storage) combination other than Cpu/Cpu, Metal/Metal, or Cuda/Cuda hits this catch-all bail.","triggerScenarios":"Calling quantize on a QTensor whose storage is on a different device than the source tensor (e.g. QStorage::Cpu quantizing a Metal f32 tensor, or vice versa).","commonSituations":"Quantizing a model checkpoint that was loaded on GPU without first moving tensors to CPU (or matching devices); forgetting .to_device() after changing the default device.","solutions":["Move the source tensor to the same device as the quantized storage before quantizing (usually cpu to_device(Device::Cpu)).","Create the destination QTensor on the same device as the source.","Check tensor.device() on both sides before calling quantize."],"exampleFix":"// before\nlet q = QTensor::quantize(&tensor_on_metal, GgmlType::Q4K, &Device::Cpu)?;\n// after\nlet tensor = tensor_on_metal.to_device(&Device::Cpu)?;\nlet q = QTensor::quantize(&tensor, GgmlType::Q4K, &Device::Cpu)?;","handlingStrategy":"validation","validationCode":"fn ensure_same_device(q: &candle_core::quantized::QTensor, src: &candle_core::Tensor) -> candle_core::Result<()> {\n    if q.device().location() != src.device().location() {\n        candle_core::bail!(\"quantize: device mismatch {:?} vs {:?}\", q.device(), src.device());\n    }\n    Ok(())\n}","typeGuard":"fn same_device(a: &candle_core::Device, b: &candle_core::Device) -> bool { a.location() == b.location() }","tryCatchPattern":null,"preventionTips":["Always to_device() the source before quantizing","Create the QTensor on the same device as the source f32 tensor","Standardize on CPU for quantization pipelines, then move results"],"tags":["device-mismatch","quantized","cpu","metal","cuda"],"backgroundTag":"device-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}