deepseek-ai/DeepSeek-V3 · error · AssertionError

Last dimension size must be divisible by block_size (block_s

Error message

Last dimension size must be divisible by block_size (block_size=${block_size})

What it means

Thrown by act_quant (inference/kernel.py:52): block-wise quantization groups exactly block_size (default 128) elements of the LAST dimension into one scaling factor (s has shape x.size(-1) // block_size), so a partial block is unrepresentable. The Triton kernel indexes blocks assuming whole multiples.

Source

Thrown at inference/kernel.py:52

    tl.store(y_ptr + offs, y)
    tl.store(s_ptr + pid, s)


def act_quant(x: torch.Tensor, block_size: int = 128, scale_fmt: Optional[str] = None) -> Tuple[torch.Tensor, torch.Tensor]:
    """
    Quantizes the input tensor `x` using block-wise quantization.

    Args:
        x (torch.Tensor): The input tensor to be quantized. Must be contiguous and its last dimension size must be divisible by `block_size`.
        block_size (int, optional): The size of the blocks to be used for quantization. Default is 128.
        scale_fmt (Optional[str], optional): The format of the scale. Default is None.
    Returns:
        Tuple[torch.Tensor, torch.Tensor]: A tuple containing:
            - The quantized tensor with dtype `torch.float8_e4m3fn`.
            - A tensor of scaling factors with dtype `torch.float32`.
    """
    assert x.is_contiguous(), 'Input tensor must be contiguous'
    assert x.size(-1) % block_size == 0, f'Last dimension size must be divisible by block_size (block_size={block_size})'
    y = torch.empty_like(x, dtype=torch.float8_e4m3fn)
    s = x.new_empty(*x.size()[:-1], x.size(-1) // block_size, dtype=torch.float32)
    grid = lambda meta: (triton.cdiv(x.numel(), meta['BLOCK_SIZE']), )
    act_quant_kernel[grid](x, y, s, BLOCK_SIZE=block_size, scale_fmt=scale_fmt)
    return y, s


@triton.jit
def weight_dequant_kernel(x_ptr, s_ptr, y_ptr, M, N, BLOCK_SIZE: tl.constexpr):
    """
    Dequantizes weights using the provided scaling factors and stores the result.

    Args:
        x_ptr (tl.pointer): Pointer to the quantized weights.
        s_ptr (tl.pointer): Pointer to the scaling factors.
        y_ptr (tl.pointer): Pointer to the output buffer for dequantized weights.
        M (int): Number of rows in the weight matrix.
        N (int): Number of columns in the weight matrix.

View on GitHub (pinned to 9b4e9788e4)

Solutions

1. Pass a tensor whose last dim is a multiple of block_size (pad with F.pad if you control the producer)
2. Verify which tensor you are quantizing — the bug is often quantizing the wrong operand (e.g. b instead of b.T)
3. Choose a block_size that divides x.size(-1), if you control block_size

Example fix

# before — 6000 not divisible by 128
y, s = act_quant(x)  # x.shape[-1] == 6000

# after
import torch.nn.functional as F
pad = (-x.size(-1)) % 128
if pad:
    x = F.pad(x, (0, pad))
y, s = act_quant(x)

BLOCK = 128
assert x.size(-1) % BLOCK == 0, (
    f"last dim {x.size(-1)} not divisible by block_size {BLOCK}; "
    f"pad the tensor or fix the producing layer"
)

def quantizable_shape(x: torch.Tensor, block_size: int = 128) -> bool:
    return x.dim() >= 1 and x.size(-1) % block_size == 0

When it happens

Trigger: Calling act_quant(x) where x.shape[-1] is not a multiple of the block_size argument — e.g. a last dim of 192 with block 128, or 6000 vs 128. All stock DeepSeek-V3 dims (7168, 2048, 576, 128...) are multiples of 128, so this implies a non-standard dim or a wrong tensor was passed.

Common situations: Custom model configs with dims not aligned to 128; accidentally passing a scale tensor or a reshaped activation as x; changing block_size to something that no longer divides the dim (e.g. block_size=96).

Related errors

• Input tensor must be contiguous
• Input tensors must have 2 dimensions
• Input tensors must be contiguous
• Scaling factor tensors must be contiguous
• Warning: Missing scale_inv tensor for ${weight_name}, skippi

AI-assisted analysis of deepseek-ai/DeepSeek-V3@9b4e9788e4 (2026-08-14). Data as JSON: /api/errors/c661a3c6bfe517d0. Report an issue: GitHub.