{"record":{"id":"207546133c7def2b","repo":"sgl-project/sglang","slug":"gguf-tensor-tensor-name-has-inner-dimension-inn","errorCode":null,"errorMessage":"GGUF tensor {tensor.name} has inner dimension {inner_dim}, which is not a multiple of block size {block_size}","messagePattern":"GGUF tensor (.+?) has inner dimension (.+?), which is not a multiple of block size (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/gguf_weights.py","lineNumber":114,"sourceCode":"            raise ValueError(\n                f\"GGUF tensor {tensor.name} declares original shape \"\n                f\"{logical_shape}, which contains {math.prod(logical_shape)} \"\n                f\"elements instead of {tensor.n_elements}\"\n            )\n        is_quantized = int(weight_type) not in _UNQUANTIZED_TYPES\n        dequantize_on_load = False\n        if is_quantized:\n            if len(logical_shape) != 2 or not tensor.name.endswith(\".weight\"):\n                raise ValueError(\n                    f\"GGUF tensor {tensor.name} is quantized, but diffusion GGUF \"\n                    \"currently supports packed data only for 2D linear .weight \"\n                    \"tensors\"\n                )\n            block_size, type_size = gguf.GGML_QUANT_SIZES[weight_type]\n            inner_dim = logical_shape[-1]\n            if inner_dim % block_size:\n                if shape_field is None:\n                    raise ValueError(\n                        f\"GGUF tensor {tensor.name} has inner dimension {inner_dim}, \"\n                        f\"which is not a multiple of block size {block_size}\"\n                    )\n                dequantize_on_load = True\n                stored_shape = logical_shape\n            else:\n                stored_shape = (\n                    *logical_shape[:-1],\n                    inner_dim // block_size * type_size,\n                )\n            if (\n                int(weight_type) in _SUPER_BLOCK_DEQUANT_TYPES\n                and math.prod(logical_shape) % _GGML_SUPER_BLOCK\n            ):\n                raise ValueError(\n                    f\"GGUF tensor {tensor.name} is not aligned to \"\n                    f\"{_GGML_SUPER_BLOCK}-element super blocks\"\n                )","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/gguf_weights.py#L96-L132","documentation":"For a quantized GGUF tensor, the inner (last) dimension must be a multiple of the quantization block size (e.g. 32 for Q8_0, 256 for some FP8/Q4 variants). When the tensor has no explicit original-shape field, the loader cannot rebuild a dequantization-friendly layout, so it rejects the tensor.","triggerScenarios":"read_gguf_tensor_meta on a quantized tensor where shape_field is None and logical_shape[-1] % block_size != 0. Note: when a shape field IS present, this situation instead sets dequantize_on_load=True and does not raise.","commonSituations":"Quantized GGUF checkpoints whose inner dimensions (e.g. 320, 640 typical of diffusion models) are not multiples of the block size; GGUFs written without ComfyUI-style shape annotations; switching block-size-sensitive quant formats on an existing checkpoint.","solutions":["Use a GGUF export that includes the original-shape field so the loader can dequantize on load instead of failing","Re-quantize with a block size that divides the inner dimension (or pad the inner dimension to a block multiple before export)","If you control the writer, add the shape annotation field for each quantized tensor"],"exampleFix":"# before: Q8_0 tensor inner_dim=320, block_size=... -> 320 % block != 0, no shape field -> raise\n# after: writer adds shape field, loader dequantizes on load\ntensor.add_text(f'{name}.shape', '[320, 320]')  # ComfyUI-style annotation","handlingStrategy":"validation","validationCode":"from gguf import GGML_QUANT_SIZES\nfor t in reader.tensors:\n    if t.tensor_type not in UNQUANTIZED:\n        bs, _ = GGML_QUANT_SIZES[t.tensor_type]\n        has_shape_field = reader.get_field(f'{t.name}.shape') is not None\n        inner = t.shape[-1]\n        if inner % bs and not has_shape_field:\n            raise MisalignedQuantTensor(t.name, inner, bs)","typeGuard":"def is_block_aligned_or_annotated(reader, t, block_size: int) -> bool:\n    return t.shape[-1] % block_size == 0 or reader.get_field(f'{t.name}.shape') is not None","tryCatchPattern":null,"preventionTips":["Export GGUFs with ComfyUI-style shape annotations for all quantized tensors","Check inner dims against block size before quantizing","Document which quant types your pipeline supports"],"tags":["gguf","quantization","block-alignment","tensor-shape"],"backgroundTag":"quantization-block-misalignment","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}