sgl-project/sglang · error · ValueError

GGUF tensor {tensor.name} is not aligned to {_GGML_SUPER_BLO

Error message

GGUF tensor {tensor.name} is not aligned to {_GGML_SUPER_BLOCK}-element super blocks

What it means

Certain GGUF quant types (in _SUPER_BLOCK_DEQUANT_TYPES) rely on 256-element GGML super blocks for their dequantization path. If the total element count of the quantized tensor is not divisible by _GGML_SUPER_BLOCK (256), the super-block dequant kernel cannot be applied and the loader rejects the tensor.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/gguf_weights.py:129

            inner_dim = logical_shape[-1]
            if inner_dim % block_size:
                if shape_field is None:
                    raise ValueError(
                        f"GGUF tensor {tensor.name} has inner dimension {inner_dim}, "
                        f"which is not a multiple of block size {block_size}"
                    )
                dequantize_on_load = True
                stored_shape = logical_shape
            else:
                stored_shape = (
                    *logical_shape[:-1],
                    inner_dim // block_size * type_size,
                )
            if (
                int(weight_type) in _SUPER_BLOCK_DEQUANT_TYPES
                and math.prod(logical_shape) % _GGML_SUPER_BLOCK
            ):
                raise ValueError(
                    f"GGUF tensor {tensor.name} is not aligned to "
                    f"{_GGML_SUPER_BLOCK}-element super blocks"
                )
            stored_dtype = torch.bfloat16 if dequantize_on_load else torch.uint8
        else:
            stored_shape = logical_shape
            stored_dtype = {
                _GGML_F32: torch.float32,
                _GGML_F16: torch.float16,
                _GGML_BF16: torch.bfloat16,
            }[int(weight_type)]

        param_name = (
            f"{tensor.name.removesuffix('.weight')}.qweight"
            if is_quantized and not dequantize_on_load
            else tensor.name
        )
        metadata[tensor.name] = GGUFTensorMeta(

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize so each affected tensor's total element count is a multiple of 256 (usually by choosing inner dims divisible by 256)
2. Use a quant type not in _SUPER_BLOCK_DEQUANT_TYPES that this loader supports natively
3. Report/fix the export tool that produced a super-block format tensor without super-block alignment

Example fix

# before: super-block type with 320*320=102400 elements? ok; 11520 elements -> 11520 % 256 != 0 -> raise
# after: pad or choose inner dim divisible so prod % 256 == 0
inner = ((inner + 255) // 256) * 256  # at export time

SUPER = 256
for t in reader.tensors:
    if t.tensor_type in SUPER_BLOCK_DEQUANT_TYPES:
        assert math.prod(t.shape) % SUPER == 0, (t.name, math.prod(t.shape))

def super_block_ok(t, types: set, super_size: int = 256) -> bool:
    return t.tensor_type not in types or math.prod(t.shape) % super_size == 0

When it happens

Trigger: read_gguf_tensor_meta on a tensor whose weight_type is in _SUPER_BLOCK_DEQUANT_TYPES and math.prod(logical_shape) % 256 != 0. E.g. an MLX-style quantized tensor with total elements not a multiple of 256.

Common situations: Diffusion GGUF checkpoints quantized with MLX-derived super-block formats whose tensor sizes (rows x cols) don't multiply to a multiple of 256; mixed checkpoints partially converted between quant formats.

Related errors

• The input size is not aligned with the quantized weight shap
• bitsandbytes 4-bit TP shard is not aligned to quantization b
• GGUFConfig must be constructed from a GGUF checkpoint
• A GGUF encoder checkpoint cannot be combined with a second q
• Cannot parse checkpoint quantization for {component_name!r}:

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fe7920600898e832. Report an issue: GitHub.