{"record":{"id":"02c238b13becd55f","repo":"huggingface/candle","slug":"quantized-tensor-has-bytes-expected-expected","errorCode":null,"errorMessage":"quantized tensor has {} bytes, expected {expected_size}","messagePattern":"quantized tensor has (.+?) bytes, expected (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/cuda.rs","lineNumber":830,"sourceCode":"    pub fn embedding(\n        &self,\n        rows: usize,\n        hidden: usize,\n        ids: &CudaStorage,\n        ids_l: &crate::Layout,\n    ) -> Result<CudaStorage> {\n        if !ids_l.is_contiguous() {\n            crate::bail!(\"quantized embedding requires contiguous ids\")\n        }\n        if !hidden.is_multiple_of(self.dtype.block_size()) {\n            crate::bail!(\n                \"quantized embedding hidden size {hidden} is not divisible by block size {}\",\n                self.dtype.block_size()\n            )\n        }\n        let expected_size = rows * hidden * self.dtype.type_size() / self.dtype.block_size();\n        if self.storage_size_in_bytes() != expected_size {\n            crate::bail!(\n                \"quantized tensor has {} bytes, expected {expected_size}\",\n                self.storage_size_in_bytes()\n            )\n        }\n        let ids = ids.as_cuda_slice::<u32>()?;\n        let ids = match ids_l.contiguous_offsets() {\n            Some((o1, o2)) => ids.slice(o1..o2),\n            None => Err(crate::Error::RequiresContiguous {\n                op: \"quantized-embedding\",\n            }\n            .bt())?,\n        };\n        get_rows(&self.data, self.dtype, hidden, &ids, self.device())\n    }\n\n    pub fn fwd(\n        &self,\n        self_shape: &crate::Shape,","sourceCodeStart":812,"sourceCodeEnd":848,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/cuda.rs#L812-L848","documentation":"After checking hidden divisibility, the embedding path validates that the quantized storage's byte size equals the exact expected size: rows * hidden * type_size / block_size. A mismatch means the storage is truncated, oversized, or was produced for a different shape — i.e. the weight tensor's shape and its quantized buffer disagree, so indexing would read out of bounds.","triggerScenarios":"Calling quantized embedding on a QTensor whose storage_size_in_bytes() does not match rows*hidden derived from the table shape — typically after loading a corrupted/partial GGUF, reshaping a quantized tensor incorrectly, or mismatching rows/hidden arguments with the actual weight.","commonSituations":"Hand-editing or truncating GGUF files; loading a model checkpoint with a config whose vocab/hidden dims differ from the file; constructing QTensor manually with the wrong dims.","solutions":["Re-download / regenerate the GGUF model file — the file is likely corrupted or truncated.","Verify the rows and hidden arguments match the actual embedding weight shape (vocab_size × dim) in the model config.","Do not reshape or slice a quantized tensor; dequantize first, reshape, then re-quantize.","Re-quantize the embedding table so storage size matches its shape."],"exampleFix":"// before: rows/hidden from a mismatched config\nlet out = qtable.embedding(cfg.hidden, &ids)?; // rows=32000, but file was 32768\n// after: derive from the weight itself and verify integrity of the model file\nlet (rows, hidden) = { /* from cfg.vocab_size, cfg.dim matching the gguf */ };","handlingStrategy":"validation","validationCode":"let expected = rows * hidden * qweight.dtype().type_size() / qweight.dtype().block_size();\nif qweight.storage_size_in_bytes() != expected {\n    anyhow::bail!(\"corrupt/reshaped quantized embedding: {} bytes, expected {expected}\",\n        qweight.storage_size_in_bytes());\n}","typeGuard":null,"tryCatchPattern":"match qweight.embedding(hidden, &ids) {\n    Err(e) if e.to_string().contains(\"bytes, expected\") => {\n        anyhow::bail!(\"model file corrupted or reshaped; re-download the GGUF: {e}\")\n    }\n    r => r?,\n}","preventionTips":["Verify GGUF checksums after download","Never reshape/slice quantized tensors — dequantize first","Keep rows/hidden sourced from the same metadata as the weight tensor"],"tags":["cuda","quantization","embedding","corrupt-data","size-mismatch"],"backgroundTag":"tensor-size-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}