{"record":{"id":"8acbd4d9451eba46","repo":"huggingface/candle","slug":"weight-rank-must-be-4","errorCode":null,"errorMessage":"weight rank ({}) must be <= 4","messagePattern":"weight rank \\((.+?)\\) must be <= 4","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/metal.rs","lineNumber":393,"sourceCode":"\n        let last_k = dst_shape.pop().unwrap();\n        if last_k != k {\n            crate::bail!(\"input tensor {layout:?} incompatible with {:?}\", self_shape)\n        }\n        dst_shape.push(n);\n        let dst_shape = Shape::from(dst_shape);\n        let device = storage.device().clone();\n        let dst = device\n            .new_buffer_builder()\n            .with_size_for(dst_shape.elem_count(), DType::F32)\n            .with_label(\"qmatmul\")\n            .build()?;\n        let encoder = device.command_encoder()?;\n\n        assert_eq!(storage.dtype(), DType::F32);\n\n        if self_shape.rank() > 4 {\n            crate::bail!(\"weight rank ({}) must be <= 4\", self_shape.rank())\n        }\n        let src0_l = crate::Layout::contiguous(\n            [vec![1; 4 - self_shape.rank()], self_shape.dims().to_vec()].concat(),\n        );\n        let src0_stride = src0_l\n            .stride()\n            .iter()\n            .map(|x| {\n                (*x as f32 * (self.dtype.type_size() as f32 / self.dtype.block_size() as f32))\n                    as usize\n            })\n            .collect::<Vec<_>>();\n\n        if src_shape.rank() > 4 {\n            crate::bail!(\"weight rank ({}) must be <= 4\", src_shape.rank())\n        }\n        let src1_l = crate::Layout::contiguous(\n            [vec![1; 4 - src_shape.rank()], src_shape.dims().to_vec()].concat(),","sourceCodeStart":375,"sourceCodeEnd":411,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/metal.rs#L375-L411","documentation":"The Metal quantized matmul kernel pads the weight layout into a fixed 4D contiguous layout; tensors with rank greater than 4 cannot be expressed, so this is raised when the quantized weight shape rank exceeds 4.","triggerScenarios":"QMatMul::fwd on Metal with a quantized weight whose shape has more than 4 dimensions (self_shape.rank() > 4).","commonSituations":"Constructing 5D+ quantized weights for exotic batching (e.g. video attention with extra head/tile dims); reshaping a quantized tensor and accidentally keeping quant blocks as a dimension.","solutions":["Reshape the weight to at most 4D before quantization or before the matmul.","Merge extra leading dimensions into a single batch dimension (e.g. [a,b,c,m,k] -> [a*b*c,m,k]).","Perform the op on CPU/CUDA if you genuinely need higher-rank quantized weights."],"exampleFix":"// before\nlet w = qtensor.reshape((1, 2, 3, 4, 768 * 4))?; // rank 5\n// after\nlet w = qtensor.reshape((6, 768 * 4))?; // merge leading dims, rank <= 4","handlingStrategy":"validation","validationCode":"fn ensure_weight_rank(w: &candle_core::quantized::QTensor) -> candle_core::Result<()> {\n    if w.rank() > 4 { candle_core::bail!(\"weight rank {} > 4\", w.rank()); }\n    Ok(())\n}","typeGuard":"fn weight_rank_ok(rank: usize) -> bool { rank <= 4 }","tryCatchPattern":null,"preventionTips":["Keep quantized weights at rank <= 4 on Metal","Merge leading batch dims instead of adding new axes","Reshape to 2D before quantizing when possible"],"tags":["metal","quantized","rank-limit"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}