{"record":{"id":"c661a3c6bfe517d0","repo":"deepseek-ai/DeepSeek-V3","slug":"last-dimension-size-must-be-divisible-by-block-siz","errorCode":null,"errorMessage":"Last dimension size must be divisible by block_size (block_size=${block_size})","messagePattern":"Last dimension size must be divisible by block_size \\(block_size=(.+?)\\)","errorType":"validation","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"inference/kernel.py","lineNumber":52,"sourceCode":"    tl.store(y_ptr + offs, y)\n    tl.store(s_ptr + pid, s)\n\n\ndef act_quant(x: torch.Tensor, block_size: int = 128, scale_fmt: Optional[str] = None) -> Tuple[torch.Tensor, torch.Tensor]:\n    \"\"\"\n    Quantizes the input tensor `x` using block-wise quantization.\n\n    Args:\n        x (torch.Tensor): The input tensor to be quantized. Must be contiguous and its last dimension size must be divisible by `block_size`.\n        block_size (int, optional): The size of the blocks to be used for quantization. Default is 128.\n        scale_fmt (Optional[str], optional): The format of the scale. Default is None.\n    Returns:\n        Tuple[torch.Tensor, torch.Tensor]: A tuple containing:\n            - The quantized tensor with dtype `torch.float8_e4m3fn`.\n            - A tensor of scaling factors with dtype `torch.float32`.\n    \"\"\"\n    assert x.is_contiguous(), 'Input tensor must be contiguous'\n    assert x.size(-1) % block_size == 0, f'Last dimension size must be divisible by block_size (block_size={block_size})'\n    y = torch.empty_like(x, dtype=torch.float8_e4m3fn)\n    s = x.new_empty(*x.size()[:-1], x.size(-1) // block_size, dtype=torch.float32)\n    grid = lambda meta: (triton.cdiv(x.numel(), meta['BLOCK_SIZE']), )\n    act_quant_kernel[grid](x, y, s, BLOCK_SIZE=block_size, scale_fmt=scale_fmt)\n    return y, s\n\n\n@triton.jit\ndef weight_dequant_kernel(x_ptr, s_ptr, y_ptr, M, N, BLOCK_SIZE: tl.constexpr):\n    \"\"\"\n    Dequantizes weights using the provided scaling factors and stores the result.\n\n    Args:\n        x_ptr (tl.pointer): Pointer to the quantized weights.\n        s_ptr (tl.pointer): Pointer to the scaling factors.\n        y_ptr (tl.pointer): Pointer to the output buffer for dequantized weights.\n        M (int): Number of rows in the weight matrix.\n        N (int): Number of columns in the weight matrix.","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/kernel.py#L34-L70","documentation":"Thrown by act_quant (inference/kernel.py:52): block-wise quantization groups exactly block_size (default 128) elements of the LAST dimension into one scaling factor (s has shape x.size(-1) // block_size), so a partial block is unrepresentable. The Triton kernel indexes blocks assuming whole multiples.","triggerScenarios":"Calling act_quant(x) where x.shape[-1] is not a multiple of the block_size argument — e.g. a last dim of 192 with block 128, or 6000 vs 128. All stock DeepSeek-V3 dims (7168, 2048, 576, 128...) are multiples of 128, so this implies a non-standard dim or a wrong tensor was passed.","commonSituations":"Custom model configs with dims not aligned to 128; accidentally passing a scale tensor or a reshaped activation as x; changing block_size to something that no longer divides the dim (e.g. block_size=96).","solutions":["Pass a tensor whose last dim is a multiple of block_size (pad with F.pad if you control the producer)","Verify which tensor you are quantizing — the bug is often quantizing the wrong operand (e.g. b instead of b.T)","Choose a block_size that divides x.size(-1), if you control block_size"],"exampleFix":"# before — 6000 not divisible by 128\ny, s = act_quant(x)  # x.shape[-1] == 6000\n\n# after\nimport torch.nn.functional as F\npad = (-x.size(-1)) % 128\nif pad:\n    x = F.pad(x, (0, pad))\ny, s = act_quant(x)","handlingStrategy":"validation","validationCode":"BLOCK = 128\nassert x.size(-1) % BLOCK == 0, (\n    f\"last dim {x.size(-1)} not divisible by block_size {BLOCK}; \"\n    f\"pad the tensor or fix the producing layer\"\n)","typeGuard":"def quantizable_shape(x: torch.Tensor, block_size: int = 128) -> bool:\n    return x.dim() >= 1 and x.size(-1) % block_size == 0","tryCatchPattern":null,"preventionTips":["Keep all quantized dims multiples of 128 in custom configs","Double-check you are passing the activation/weight operand, not a scale or wrong tensor","If padding, remember to slice the GEMM result back after dequantized math"],"tags":["triton","fp8","quantization","shape-validation","deepseek"],"backgroundTag":null,"analyzedSha":"9b4e9788e4a3a731f7567338ed15d3ec549ce03b","analyzedAt":"2026-08-14T19:02:32.748Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}