deepseek-ai/DeepSeek-V3 · error · AssertionError

Scaling factor tensors must be contiguous

Error message

Scaling factor tensors must be contiguous

What it means

Thrown by fp8_gemm (inference/kernel.py:189): the per-block scaling factors a_s and b_s are also passed to the Triton kernel as raw pointers and must be contiguous. They are produced by act_quant (contiguous by construction), so this assert typically fires only when scales are sliced, transposed, or otherwise re-viewed by custom code.

Source

Thrown at inference/kernel.py:189

    mask = (offs_m[:, None] < M) & (offs_n[None, :] < N)
    tl.store(c_ptrs, c, mask=mask)


def fp8_gemm(a: torch.Tensor, a_s: torch.Tensor, b: torch.Tensor, b_s: torch.Tensor):
    """
    Perform a matrix multiplication using FP8 precision.

    Args:
        a (torch.Tensor): The first input matrix, must be contiguous.
        a_s (torch.Tensor): The scaling factor for the first input matrix, must be contiguous.
        b (torch.Tensor): The second input matrix, must be contiguous.
        b_s (torch.Tensor): The scaling factor for the second input matrix, must be contiguous.

    Returns:
        torch.Tensor: The result of the matrix multiplication.
    """
    assert a.is_contiguous() and b.is_contiguous(), 'Input tensors must be contiguous'
    assert a_s.is_contiguous() and b_s.is_contiguous(), 'Scaling factor tensors must be contiguous'
    K = a.size(-1)
    M = a.numel() // K
    N = b.size(0)
    c = a.new_empty(*a.size()[:-1], N, dtype=torch.get_default_dtype())
    grid = lambda META: (triton.cdiv(M, META['BLOCK_SIZE_M']), triton.cdiv(N, META['BLOCK_SIZE_N']))
    fp8_gemm_kernel[grid](a, b, c, a_s, b_s, M, N, K)
    return c

View on GitHub (pinned to 9b4e9788e4)

Solutions

1. Pass scales directly from act_quant output without modification
2. If you must transform them, end with .contiguous(): a_s = a_s.transpose(-1,-2).contiguous()
3. Check that sharding code materializes per-rank scale slices with .contiguous() (as convert.py does)

Example fix

# before
out = fp8_gemm(a, a_s[:, 1:], b, b_s)  # sliced scale — view

# after
a_s = a_s[:, 1:].contiguous()
out = fp8_gemm(a, a_s, b, b_s)

if not (a_s.is_contiguous() and b_s.is_contiguous()):
    a_s, b_s = a_s.contiguous(), b_s.contiguous()
c = fp8_gemm(a, a_s, b, b_s)

def scales_ready(a_s: torch.Tensor, b_s: torch.Tensor) -> bool:
    return a_s.is_contiguous() and b_s.is_contiguous()

When it happens

Trigger: Calling fp8_gemm with a_s/b_s obtained via slicing (e.g. s[..., 1:]), transpose, or narrow of an act_quant output, or scales loaded from a checkpoint sharded with views. The preceding assert (error 11) checks a/b; this one covers the scale tensors.

Common situations: Custom kernels/fusers that manipulate scale tensors for batching; multi-rank setups passing per-rank scale shards as views instead of copies.

Related errors

• Input tensor must be contiguous
• Input tensors must be contiguous
• Last dimension size must be divisible by block_size (block_s
• Input tensors must have 2 dimensions
• Warning: Missing scale_inv tensor for ${weight_name}, skippi

AI-assisted analysis of deepseek-ai/DeepSeek-V3@9b4e9788e4 (2026-08-14). Data as JSON: /api/errors/490c6a9e852f5e2f. Report an issue: GitHub.