sgl-project/sglang · error · ValueError

GGUF tensor {tensor.name} is quantized, but diffusion GGUF c

Error message

GGUF tensor {tensor.name} is quantized, but diffusion GGUF currently supports packed data only for 2D linear .weight tensors

What it means

The GGUF loader supports packed (still-quantized) data only for 2D linear-layer .weight tensors. This tensor is quantized but is either not 2D or not named *.weight (e.g. a conv weight, bias, or norm tensor), so there is no packed layout it can safely map.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/gguf_weights.py:105

    for tensor in reader.tensors:
        weight_type = WeightType(tensor.tensor_type)
        shape_field = reader.fields.get(f"comfy.gguf.orig_shape.{tensor.name}")
        logical_shape = (
            tuple(int(dim) for dim in shape_field.contents())
            if shape_field is not None
            else tuple(int(dim) for dim in reversed(tensor.shape))
        )
        if math.prod(logical_shape) != tensor.n_elements:
            raise ValueError(
                f"GGUF tensor {tensor.name} declares original shape "
                f"{logical_shape}, which contains {math.prod(logical_shape)} "
                f"elements instead of {tensor.n_elements}"
            )
        is_quantized = int(weight_type) not in _UNQUANTIZED_TYPES
        dequantize_on_load = False
        if is_quantized:
            if len(logical_shape) != 2 or not tensor.name.endswith(".weight"):
                raise ValueError(
                    f"GGUF tensor {tensor.name} is quantized, but diffusion GGUF "
                    "currently supports packed data only for 2D linear .weight "
                    "tensors"
                )
            block_size, type_size = gguf.GGML_QUANT_SIZES[weight_type]
            inner_dim = logical_shape[-1]
            if inner_dim % block_size:
                if shape_field is None:
                    raise ValueError(
                        f"GGUF tensor {tensor.name} has inner dimension {inner_dim}, "
                        f"which is not a multiple of block size {block_size}"
                    )
                dequantize_on_load = True
                stored_shape = logical_shape
            else:
                stored_shape = (
                    *logical_shape[:-1],
                    inner_dim // block_size * type_size,

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize the checkpoint leaving non-linear tensors (conv, norm, bias) in an unquantized type such as F32/F16/BF16
2. If the tensor genuinely is a linear weight, check that its name ends with '.weight' and its declared shape is 2D; fix naming/shape metadata
3. Use a checkpoint flavor known to work with this loader (only 2D linear weights quantized)

Example fix

# before: quantize everything, including convs -> error on 'conv_in.weight' [320,4,3,3]
# after (quantizer config): exclude conv/norm layers
quant_config.exclude_patterns = ['conv*', '*norm*']  # keep them F16

for t in reader.tensors:
    wt = reader.get_field('general.type')  # per-tensor type from t.tensor_type
    if t.tensor_type not in UNQUANTIZED and (len(t.shape) != 2 or not t.name.endswith('.weight')):
        raise SkipOrRequantize(t.name)

def is_packed_safe(t) -> bool:
    return t.tensor_type in UNQUANTIZED or (len(t.shape) == 2 and t.name.endswith('.weight'))

try:
    meta = read_gguf_tensor_meta(reader, t)
except ValueError as e:
    if 'packed data only' in str(e):
        log.warning('skipping unsupported quantized tensor %s', t.name)
    else:
        raise

When it happens

Trigger: read_gguf_tensor_meta encounters a tensor with weight_type not in _UNQUANTIZED_TYPES whose logical_shape has rank != 2 or whose name does not end with '.weight'. Common with quantized conv/norm tensors in diffusion GGUF checkpoints.

Common situations: Loading a ComfyUI diffusion GGUF where convolution or group-norm tensors were quantized alongside linear weights; using a quantizer preset that quantizes non-linear layers; a GGUF produced for a different runtime that allows packed N-D tensors.

Related errors

• GGUFConfig must be constructed from a GGUF checkpoint
• A GGUF encoder checkpoint cannot be combined with a second q
• Cannot parse checkpoint quantization for {component_name!r}:
• GGUF tensor {tensor.name} declares original shape {logical_s
• GGUF tensor {tensor.name} has inner dimension {inner_dim}, w

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/df75263b47c2bb4a. Report an issue: GitHub.