{"record":{"id":"1726db5f613dff3e","repo":"huggingface/candle","slug":"quantized-embedding-hidden-size-hidden-is-not-di","errorCode":null,"errorMessage":"quantized embedding hidden size {hidden} is not divisible by block size {}","messagePattern":"quantized embedding hidden size (.+?) is not divisible by block size (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/cuda.rs","lineNumber":823,"sourceCode":"        Ok(())\n    }\n\n    pub fn storage_size_in_bytes(&self) -> usize {\n        self.data.len\n    }\n\n    pub fn embedding(\n        &self,\n        rows: usize,\n        hidden: usize,\n        ids: &CudaStorage,\n        ids_l: &crate::Layout,\n    ) -> Result<CudaStorage> {\n        if !ids_l.is_contiguous() {\n            crate::bail!(\"quantized embedding requires contiguous ids\")\n        }\n        if !hidden.is_multiple_of(self.dtype.block_size()) {\n            crate::bail!(\n                \"quantized embedding hidden size {hidden} is not divisible by block size {}\",\n                self.dtype.block_size()\n            )\n        }\n        let expected_size = rows * hidden * self.dtype.type_size() / self.dtype.block_size();\n        if self.storage_size_in_bytes() != expected_size {\n            crate::bail!(\n                \"quantized tensor has {} bytes, expected {expected_size}\",\n                self.storage_size_in_bytes()\n            )\n        }\n        let ids = ids.as_cuda_slice::<u32>()?;\n        let ids = match ids_l.contiguous_offsets() {\n            Some((o1, o2)) => ids.slice(o1..o2),\n            None => Err(crate::Error::RequiresContiguous {\n                op: \"quantized-embedding\",\n            }\n            .bt())?,","sourceCodeStart":805,"sourceCodeEnd":841,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/cuda.rs#L805-L841","documentation":"Quantized embedding kernels process the hidden dimension in fixed block-size chunks of the quantized format (e.g. 32 for Q8_0/Q4K group quants). If the embedding table's hidden size is not a multiple of the dtype's block size, rows cannot be indexed/dequantized safely, so candle bails with the hidden size and block size in the message.","triggerScenarios":"Calling quantized embedding where the number of columns of the embedding weight (hidden) is not divisible by dtype.block_size() — e.g. hidden=100 with a block size of 32.","commonSituations":"Quantizing a model with an unusual/legacy embedding dimension; changing the model config (dim) without adjusting it to the quant block size; custom architectures with non-standard hidden sizes.","solutions":["Adjust the model hidden size so it is divisible by the quant dtype's block size (e.g. 32 or 256 for K-quants).","Choose a quantized dtype whose block size divides the hidden size (e.g. Q8_0 vs a K-quant with different block layout).","Keep the embedding layer unquantized (f32/f16) and quantize only other layers.","Pad rows to a compatible size — requires re-quantizing the padded table and slicing outputs back."],"exampleFix":"// before (hidden = 100, Q4K block size 256)\nlet q = qtensor.embedding(h, &ids)?;\n// after: pick a config where hidden % block_size == 0, or leave embedding unquantized\nlet q = weight.to_dtype(candle_core::DType::F32)?.embedding(h, &ids)?;","handlingStrategy":"validation","validationCode":"let block = qweight.dtype().block_size();\nassert!(hidden % block == 0,\n    \"hidden {hidden} not divisible by block size {block}; adjust model config or dtype\");\nlet out = qweight.embedding(h, &ids)?;","typeGuard":null,"tryCatchPattern":"match qweight.embedding(hidden, &ids) {\n    Err(e) if e.to_string().contains(\"not divisible by block size\") => {\n        // fallback: dequantize and use dense embedding\n        let w = qweight.dequantize(&device)?;\n        w.embedding(&ids)?\n    }\n    r => r?,\n}","preventionTips":["Choose hidden sizes divisible by the quant block size (e.g. 32/256) in model configs","Leave embedding tables in f32/f16 when dims are incompatible","Check block_size() compatibility when selecting a GGUF quant variant"],"tags":["cuda","quantization","embedding","shape","block-size"],"backgroundTag":"shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}