{"record":{"id":"4c1ad0cbdf18bbc2","repo":"sgl-project/sglang","slug":"gptq-act-order-on-xpu-requires-each-group-size-blo","errorCode":null,"errorMessage":"GPTQ act_order on XPU requires each group_size block of input channels to map to a single group, but this shard splits a group across the K boundary.{tp_hint}","messagePattern":"GPTQ act_order on XPU requires each group_size block of input channels to map to a single group, but this shard splits a group across the K boundary\\.(.+?)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/hardware_backend/xpu/quantization/gptq_kernels.py","lineNumber":71,"sourceCode":"            g_idx = layer.g_idx.data\n            if g_idx.numel() != k:\n                raise ValueError(\n                    \"GPTQ act_order on XPU expects a per-channel g_idx of length \"\n                    f\"K={k}, got {g_idx.numel()}.\"\n                )\n            # Sort K by group id so groups become contiguous gs-blocks.\n            act_perm = torch.argsort(g_idx, stable=True).to(torch.int64)\n            codes = codes[act_perm, :]\n            sorted_g = g_idx[act_perm].to(torch.int64)\n            blocks = sorted_g.view(-1, group_size)\n            if not torch.equal(blocks, blocks[:, :1].expand_as(blocks)):\n                tp_size = get_parallel().tp_size\n                tp_hint = (\n                    f\" Got tp_size={tp_size}; please use --tp-size 1.\"\n                    if tp_size > 1\n                    else \"\"\n                )\n                raise NotImplementedError(\n                    \"GPTQ act_order on XPU requires each group_size block of \"\n                    \"input channels to map to a single group, but this shard \"\n                    \"splits a group across the K boundary.\" + tp_hint\n                )\n            # Reorder scales/zeros to follow the block group order.\n            block_gid = blocks[:, 0]  # [num_blocks]\n            scales = scales[block_gid]\n            zp = zp[block_gid]\n\n        codes = codes.t().contiguous()  # [N, K]\n        qweight_uint8 = pack_int4_to_uint8(codes)  # [N, K // 2]\n        qweight_packed = torch.ops.aten._convert_weight_to_int4pack(\n            qweight_uint8, 8\n        )  # [N, K // 8] int32\n\n        replace_parameter(layer, \"qweight\", qweight_packed)\n        layer.register_parameter(\n            \"xpu_scales\",","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/hardware_backend/xpu/quantization/gptq_kernels.py#L53-L89","documentation":"XPU GPTQ kernels with act_order (gidx) require each group_size block of input channels to stay within one quantization group after tensor-parallel sharding. When TP sharding splits the K dimension such that a group straddles the shard boundary, per-group reordering cannot be applied correctly, so weight post-processing raises NotImplementedError. The error message includes the detected tp_size and suggests running unsharded.","triggerScenarios":"Loading a GPTQ checkpoint with act_order=True (desc_act) on Intel XPU with --tp-size > 1 where (input_channels / tp_size) is not aligned to the checkpoint's group_size (e.g. group_size=128 and the per-shard K is not a multiple of 128). Raised from process_weights_after_loading in gptq_kernels.py:71 during model weight loading.","commonSituations":"Running a GPTQ act-order model (e.g. TheBloke GPTQ with desc_act) on Intel GPUs with tensor parallelism where K is not divisible by tp_size*group_size; switching a working single-GPU config to multi-GPU TP; newer checkpoints with unusual group_size values.","solutions":["Use --tp-size 1 (as the tp_hint suggests) so the full K dimension lives in one shard","Pick a tp_size such that (num_input_channels / tp_size) is an integer multiple of the checkpoint's group_size","Use a GPTQ checkpoint without act_order/desc_act, or a quantization format supported on XPU with TP (e.g. unsharded-compatible formats)","Report/request upstream support for boundary-splitting groups in the XPU act_order path"],"exampleFix":"# before\npython -m sglang.launch_server --model gptq-model --tp-size 4  # raises NotImplementedError\n\n# after\npython -m sglang.launch_server --model gptq-model --tp-size 1","handlingStrategy":"validation","validationCode":"group_size = ckpt_quant_cfg.group_size  # e.g. 128\nk = num_input_channels\ntp = get_parallel().tp_size\nif act_order and tp > 1 and (k // tp) % group_size != 0:\n    raise SystemExit(\n        f\"GPTQ act_order on XPU with tp={tp} splits a group \"\n        f\"(K/tp={k//tp}, group_size={group_size}); use --tp-size 1\"\n    )","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check (K / tp_size) % group_size == 0 before enabling TP for act-order GPTQ models on XPU","Prefer tp_size=1 or group-aligned tp_size for desc_act GPTQ checkpoints","Validate quant config (group_size, act_order) at checkpoint load time in CI"],"tags":["xpu","gptq","quantization","tensor-parallel","act-order","not-implemented"],"backgroundTag":"tensor-parallel-sharding-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}