{"record":{"id":"ebe7cfb00163ce27","repo":"huggingface/candle","slug":"quantized-embedding-requires-contiguous-ids","errorCode":null,"errorMessage":"quantized embedding requires contiguous ids","messagePattern":"quantized embedding requires contiguous ids","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/cuda.rs","lineNumber":820,"sourceCode":"            inner,\n            len: data.len(),\n        };\n        Ok(())\n    }\n\n    pub fn storage_size_in_bytes(&self) -> usize {\n        self.data.len\n    }\n\n    pub fn embedding(\n        &self,\n        rows: usize,\n        hidden: usize,\n        ids: &CudaStorage,\n        ids_l: &crate::Layout,\n    ) -> Result<CudaStorage> {\n        if !ids_l.is_contiguous() {\n            crate::bail!(\"quantized embedding requires contiguous ids\")\n        }\n        if !hidden.is_multiple_of(self.dtype.block_size()) {\n            crate::bail!(\n                \"quantized embedding hidden size {hidden} is not divisible by block size {}\",\n                self.dtype.block_size()\n            )\n        }\n        let expected_size = rows * hidden * self.dtype.type_size() / self.dtype.block_size();\n        if self.storage_size_in_bytes() != expected_size {\n            crate::bail!(\n                \"quantized tensor has {} bytes, expected {expected_size}\",\n                self.storage_size_in_bytes()\n            )\n        }\n        let ids = ids.as_cuda_slice::<u32>()?;\n        let ids = match ids_l.contiguous_offsets() {\n            Some((o1, o2)) => ids.slice(o1..o2),\n            None => Err(crate::Error::RequiresContiguous {","sourceCodeStart":802,"sourceCodeEnd":838,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/cuda.rs#L802-L838","documentation":"The quantized CUDA embedding lookup reads token ids directly from device memory assuming a dense, contiguous layout. If the ids Layout has strides/non-standard offsets, the kernel cannot index them, so candle bails before launching. Only contiguous u32 id tensors are accepted.","triggerScenarios":"Calling QTensor::embedding (or forward on an embedding whose weight is a quantized QTensor) with an ids tensor that is a non-contiguous slice/view — e.g. the result of slicing, striding, or transposing without a contiguous copy.","commonSituations":"Passing token ids sliced out of a padded batch; building ids via indexing ops that keep a stride layout; reusing an ids layout transformed by broadcasting or narrow.","solutions":["Make the ids tensor contiguous before the call: let ids = ids.contiguous()?;","Rebuild the ids tensor freshly (e.g. via Tensor::new / cat) so it has a contiguous layout.","Avoid slicing/striding the ids on device; gather ids on CPU into a new contiguous tensor.","Check ids.layout().is_contiguous() before invoking embedding."],"exampleFix":"// before\nlet out = qweight.embedding(h, &ids_view)?;\n// after\nlet out = qweight.embedding(h, &ids_view.contiguous()?)?;","handlingStrategy":"validation","validationCode":"let ids = ids.contiguous()?; // ensure contiguous layout before quantized embedding\n// candle: assert ids is u32 on the same device\nlet ids = ids.to_dtype(candle_core::DType::U32)?.to_device(&device)?;","typeGuard":null,"tryCatchPattern":"match qweight.embedding(h, &ids) {\n    Err(e) if e.to_string().contains(\"requires contiguous ids\") => {\n        let out = qweight.embedding(h, &ids.contiguous()?)?;\n        out\n    }\n    r => r?,\n}","preventionTips":["Call .contiguous()? on ids after any slicing/narrow","Keep id tensors freshly created per batch, not as views of padded buffers","Avoid strided/transposed id layouts in tokenizers feeding quantized embedding"],"tags":["cuda","quantization","embedding","layout","contiguity"],"backgroundTag":"tensor-not-contiguous","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}