{"record":{"id":"6957d27a33dbb3b9","repo":"deepseek-ai/DeepSeek-V3","slug":"input-tensors-must-have-2-dimensions","errorCode":null,"errorMessage":"Input tensors must have 2 dimensions","messagePattern":"Input tensors must have 2 dimensions","errorType":"validation","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"inference/kernel.py","lineNumber":105,"sourceCode":"\n\ndef weight_dequant(x: torch.Tensor, s: torch.Tensor, block_size: int = 128) -> torch.Tensor:\n    \"\"\"\n    Dequantizes the given weight tensor using the provided scale tensor.\n\n    Args:\n        x (torch.Tensor): The quantized weight tensor of shape (M, N).\n        s (torch.Tensor): The scale tensor of shape (M//block_size, N//block_size).\n        block_size (int, optional): The block size to use for dequantization. Defaults to 128.\n\n    Returns:\n        torch.Tensor: The dequantized weight tensor of the same shape as `x`.\n\n    Raises:\n        AssertionError: If `x` or `s` are not contiguous or if their dimensions are not 2.\n    \"\"\"\n    assert x.is_contiguous() and s.is_contiguous(), 'Input tensors must be contiguous'\n    assert x.dim() == 2 and s.dim() == 2, 'Input tensors must have 2 dimensions'\n    M, N = x.size()\n    y = torch.empty_like(x, dtype=torch.get_default_dtype())\n    grid = lambda meta: (triton.cdiv(M, meta['BLOCK_SIZE']), triton.cdiv(N, meta['BLOCK_SIZE']))\n    weight_dequant_kernel[grid](x, s, y, M, N, BLOCK_SIZE=block_size)\n    return y\n\n\nfp8_gemm_configs = [\n    Config({'BLOCK_SIZE_M': block_m, 'BLOCK_SIZE_N': block_n, 'BLOCK_SIZE_K': 128}, num_stages=num_stages, num_warps=8)\n    for block_m in [16, 32, 64] for block_n in [32, 64, 128] for num_stages in [3, 4, 5, 6]\n]\n\n@triton.autotune(configs=fp8_gemm_configs, key=['N', 'K'])\n@triton.jit\ndef fp8_gemm_kernel(a_ptr, b_ptr, c_ptr,\n                    a_s_ptr, b_s_ptr,\n                    M, N: tl.constexpr, K: tl.constexpr,\n                    BLOCK_SIZE_M: tl.constexpr,","sourceCodeStart":87,"sourceCodeEnd":123,"githubUrl":"https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/kernel.py#L87-L123","documentation":"Thrown by weight_dequant (inference/kernel.py:105): the kernel treats x as an (M, N) matrix with s of shape (M//block, N//block) and launches a 2-D tile grid, so both tensors must be exactly 2-D. Higher- or lower-rank tensors would make the block-to-scale indexing ambiguous, hence the assert.","triggerScenarios":"Calling weight_dequant on a 1-D bias/scale vector or a 3-D+ tensor (e.g. a batched weight or an unsqueezed tensor). Also fires if x and s come from mismatched checkpoints with different rank.","commonSituations":"Custom scripts iterating over ALL tensors in a state dict (including 1-D norms/biases) and calling weight_dequant unconditionally instead of only on element_size()==1 2-D weights as fp8_cast_bf16.py does.","solutions":["Only dequantize 2-D FP8 weights: guard with x.dim() == 2 and weight.element_size() == 1","Flatten/reshape deliberately if you truly have a higher-rank weight: x = x.reshape(-1, x.size(-1)) with matching scale reshape","Skip 1-D tensors (norms, biases) — they are stored in bf16 already"],"exampleFix":"# before — dequant everything in the dict\nfor name, t in state_dict.items():\n    out[name] = weight_dequant(t, scales[name])\n\n# after\nfor name, t in state_dict.items():\n    if t.dim() == 2 and t.element_size() == 1:\n        out[name] = weight_dequant(t, scales[f'{name}_scale_inv'])\n    else:\n        out[name] = t","handlingStrategy":"validation","validationCode":"assert x.dim() == 2 and s.dim() == 2, (\n    f\"weight_dequant needs 2-D tensors, got x.dim()={x.dim()}, s.dim()={s.dim()}; \"\n    f\"skip 1-D norms/biases or reshape batched weights\"\n)","typeGuard":"def is_dequantizable_weight(t: torch.Tensor) -> bool:\n    return t.dim() == 2 and t.element_size() == 1  # 2-D FP8 weight","tryCatchPattern":null,"preventionTips":["Filter state dicts to 2-D, element_size()==1 tensors before dequantizing","Keep 1-D params (norms, biases) untouched — they are not FP8-block-quantized","Reshape deliberately, with the scale tensor reshaped to match"],"tags":["triton","fp8","dequantization","shape-validation","deepseek"],"backgroundTag":null,"analyzedSha":"9b4e9788e4a3a731f7567338ed15d3ec549ce03b","analyzedAt":"2026-08-14T19:02:32.748Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}