{"record":{"id":"132925b4fd21b9d9","repo":"sgl-project/sglang","slug":"name-must-be-a-scalar-tensor-got-shape-tuple-s","errorCode":null,"errorMessage":"{name} must be a scalar tensor, got shape {tuple(scale.shape)}","messagePattern":"(.+?) must be a scalar tensor, got shape (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":433,"sourceCode":"            raise ValueError(\n                f\"attn_sink must be on q's device {device}, got {attn_sink.device}\"\n            )\n        if not attn_sink.is_contiguous():\n            raise ValueError(\"attn_sink must be contiguous\")\n\n    for name, scale in ((\"q_scale\", q_scale), (\"kv_scale\", kv_scale)):\n        if not isinstance(scale, torch.Tensor):\n            raise ValueError(f\"{name} must be a torch.Tensor\")\n        if not scale.is_cuda:\n            raise ValueError(f\"{name} must be a CUDA tensor\")\n        if scale.device != device:\n            raise ValueError(\n                f\"{name} must be on q's device {device}, got {scale.device}\"\n            )\n        if scale.dtype != torch.float32:\n            raise ValueError(f\"{name} must be float32, got {scale.dtype}\")\n        if scale.numel() != 1:\n            raise ValueError(\n                f\"{name} must be a scalar tensor, got shape {tuple(scale.shape)}\"\n            )\n        if not scale.is_contiguous():\n            raise ValueError(f\"{name} must be contiguous\")\n\n    if out is None:\n        out = torch.empty(s_q, h_q, d_v, dtype=torch.bfloat16, device=device)\n    else:\n        _check_out_buffer(out, \"out\", (s_q, h_q, d_v), torch.bfloat16, device)\n\n    if max_logits is None:\n        max_logits = torch.empty(s_q, h_q, dtype=torch.float32, device=device)\n    else:\n        _check_out_buffer(max_logits, \"max_logits\", (s_q, h_q), torch.float32, device)\n\n    if lse is None:\n        lse = torch.empty(s_q, h_q, dtype=torch.float32, device=device)\n    else:","sourceCodeStart":415,"sourceCodeEnd":451,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L415-L451","documentation":"q_scale and kv_scale must each be a 1-element (scalar-shaped) tensor, e.g. shape () or (1,). The kernel loads a single scale value; multi-element scales would leave the de-scale factor ambiguous.","triggerScenarios":"Passing per-head or per-group scale vectors with shape (h_q,) or (num_groups,) as q_scale/kv_scale — this kernel only supports scalar (per-tensor) quantization scales.","commonSituations":"Migrating from a per-channel quantized attention backend whose scale tensors are vectors, then handing them to this per-tensor-scaled kernel unchanged.","solutions":["Reduce to a scalar: q_scale = q_scale_vector.mean().float().reshape(1) if a single value is acceptable","Use the correct per-tensor de-scale values from the quantization recipe rather than per-channel scales"],"exampleFix":"// before\nq_scale = per_channel_scales  # shape (h_q,)\n// after\nq_scale = torch.tensor(calibrated_per_tensor_scale, dtype=torch.float32, device=q.device)","handlingStrategy":"validation","validationCode":"assert q_scale.numel() == 1 and kv_scale.numel() == 1","typeGuard":"def is_scalar_tensor(t: torch.Tensor) -> bool:\n    return isinstance(t, torch.Tensor) and t.numel() == 1","tryCatchPattern":null,"preventionTips":["Confirm the quantization recipe is per-tensor before using this kernel","Reduce per-channel scales to a single calibrated value at setup"],"tags":["mla","quantization","scale-validation","shape"],"backgroundTag":"tensor-shape-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}