{"record":{"id":"fe7920600898e832","repo":"sgl-project/sglang","slug":"gguf-tensor-tensor-name-is-not-aligned-to-ggml","errorCode":null,"errorMessage":"GGUF tensor {tensor.name} is not aligned to {_GGML_SUPER_BLOCK}-element super blocks","messagePattern":"GGUF tensor (.+?) is not aligned to (.+?)-element super blocks","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/gguf_weights.py","lineNumber":129,"sourceCode":"            inner_dim = logical_shape[-1]\n            if inner_dim % block_size:\n                if shape_field is None:\n                    raise ValueError(\n                        f\"GGUF tensor {tensor.name} has inner dimension {inner_dim}, \"\n                        f\"which is not a multiple of block size {block_size}\"\n                    )\n                dequantize_on_load = True\n                stored_shape = logical_shape\n            else:\n                stored_shape = (\n                    *logical_shape[:-1],\n                    inner_dim // block_size * type_size,\n                )\n            if (\n                int(weight_type) in _SUPER_BLOCK_DEQUANT_TYPES\n                and math.prod(logical_shape) % _GGML_SUPER_BLOCK\n            ):\n                raise ValueError(\n                    f\"GGUF tensor {tensor.name} is not aligned to \"\n                    f\"{_GGML_SUPER_BLOCK}-element super blocks\"\n                )\n            stored_dtype = torch.bfloat16 if dequantize_on_load else torch.uint8\n        else:\n            stored_shape = logical_shape\n            stored_dtype = {\n                _GGML_F32: torch.float32,\n                _GGML_F16: torch.float16,\n                _GGML_BF16: torch.bfloat16,\n            }[int(weight_type)]\n\n        param_name = (\n            f\"{tensor.name.removesuffix('.weight')}.qweight\"\n            if is_quantized and not dequantize_on_load\n            else tensor.name\n        )\n        metadata[tensor.name] = GGUFTensorMeta(","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/gguf_weights.py#L111-L147","documentation":"Certain GGUF quant types (in _SUPER_BLOCK_DEQUANT_TYPES) rely on 256-element GGML super blocks for their dequantization path. If the total element count of the quantized tensor is not divisible by _GGML_SUPER_BLOCK (256), the super-block dequant kernel cannot be applied and the loader rejects the tensor.","triggerScenarios":"read_gguf_tensor_meta on a tensor whose weight_type is in _SUPER_BLOCK_DEQUANT_TYPES and math.prod(logical_shape) % 256 != 0. E.g. an MLX-style quantized tensor with total elements not a multiple of 256.","commonSituations":"Diffusion GGUF checkpoints quantized with MLX-derived super-block formats whose tensor sizes (rows x cols) don't multiply to a multiple of 256; mixed checkpoints partially converted between quant formats.","solutions":["Re-quantize so each affected tensor's total element count is a multiple of 256 (usually by choosing inner dims divisible by 256)","Use a quant type not in _SUPER_BLOCK_DEQUANT_TYPES that this loader supports natively","Report/fix the export tool that produced a super-block format tensor without super-block alignment"],"exampleFix":"# before: super-block type with 320*320=102400 elements? ok; 11520 elements -> 11520 % 256 != 0 -> raise\n# after: pad or choose inner dim divisible so prod % 256 == 0\ninner = ((inner + 255) // 256) * 256  # at export time","handlingStrategy":"validation","validationCode":"SUPER = 256\nfor t in reader.tensors:\n    if t.tensor_type in SUPER_BLOCK_DEQUANT_TYPES:\n        assert math.prod(t.shape) % SUPER == 0, (t.name, math.prod(t.shape))","typeGuard":"def super_block_ok(t, types: set, super_size: int = 256) -> bool:\n    return t.tensor_type not in types or math.prod(t.shape) % super_size == 0","tryCatchPattern":null,"preventionTips":["Verify total element counts are 256-multiples before using super-block formats","Keep a compatibility test matrix of quant type vs tensor shapes","Avoid MLX-derived formats on small odd-sized tensors"],"tags":["gguf","quantization","super-block","alignment"],"backgroundTag":"quantization-block-misalignment","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}