{"record":{"id":"7691f020f94a4880","repo":"sgl-project/sglang","slug":"unexpected-ascend-tnd-softmax-lse-shape-expected","errorCode":null,"errorMessage":"Unexpected Ascend TND softmax LSE shape: expected {(q.shape[0], q.shape[1], 1)}, got {tuple(lse.shape)}","messagePattern":"Unexpected Ascend TND softmax LSE shape: expected (.+?), got (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py","lineNumber":149,"sourceCode":"        else:\n            q, k, v = q.contiguous(), k.contiguous(), v.contiguous()\n\n    output, lse = torch.ops.npu.npu_fused_infer_attention_score(\n        q,\n        k,\n        v,\n        num_heads=q.shape[1],\n        num_key_value_heads=k.shape[1],\n        scale=q.shape[-1] ** -0.5 if softmax_scale is None else softmax_scale,\n        input_layout=\"TND\",\n        actual_seq_lengths=actual_seq_lengths,\n        actual_seq_lengths_kv=actual_seq_lengths_kv,\n        softmax_lse_flag=return_softmax_lse,\n    )\n    if not return_softmax_lse:\n        return output\n    if lse.shape != (q.shape[0], q.shape[1], 1):\n        raise RuntimeError(\n            \"Unexpected Ascend TND softmax LSE shape: \"\n            f\"expected {(q.shape[0], q.shape[1], 1)}, got {tuple(lse.shape)}\"\n        )\n    return output, lse.squeeze(-1).transpose(0, 1).contiguous()\n\n\n@dataclass\nclass AscendFAMetadata:\n    pass\n\n\nclass AscendFAMetadataBuilder(AttentionMetadataBuilder):\n    def __init__(self) -> None:\n        pass\n\n    def prepare(self) -> None:\n        pass\n","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py#L131-L167","documentation":"The Ascend TND attention backend validates the shape of the softmax LSE tensor returned by the NPU flash-attention kernel. After calling the kernel with softmax_lse_flag=True it expects lse to be [num_tokens, num_heads, 1]; any other shape means the kernel contract changed or the packed inputs were malformed.","triggerScenarios":"Calling fused_infer_attention_varlen via forward_varlen or forward_ring_kv_chunk with return_softmax_lse=True on the Ascend backend, where the NPU attentions kernel returns an LSE with unexpected rank or dim ordering.","commonSituations":"Upgrading/changing sgkernel-npu or the 'attentions' package so the kernel's LSE layout no longer matches; running ring-attention KV merge on Ascend NPU; edge-case head counts or dtypes handled differently by the kernel.","solutions":["Print tuple(lse.shape) at the raise site and compare with (q.shape[0], q.shape[1], 1) to identify the actual layout","Pin/roll back the sgkernel-npu (sgl-kernel-npu) version to one this backend was validated against","If it is a permuted/squeezed variant (e.g. [H, T]), add an adapter in the caller that reshapes to (T, H, 1) before consuming it","Report upstream with kernel version and tensor shapes if the kernel contract genuinely changed"],"exampleFix":"// before\nout, lse = ascend_backend.forward_varlen(q, k, v, cu_seqlens=cu, max_seqlen=m, return_softmax_lse=True)  # RuntimeError\n// after: adapt known alternate layout\nout, lse = ascend_backend.forward_varlen(q, k, v, cu_seqlens=cu, max_seqlen=m, return_softmax_lse=True)\nif lse.dim() == 2:  # e.g. [H, T]\n    lse = lse.transpose(0, 1).unsqueeze(-1)  # -> [T, H, 1]","handlingStrategy":"try-catch","validationCode":"assert q.ndim == 3, \"Ascend TND path requires packed [T, H, D] tensors\"\nassert q.shape[1] == k.shape[1] or q.shape[1] % k.shape[1] == 0  # sane head config","typeGuard":"def supports_ascend_lse(backend) -> bool:\n    return type(backend).forward_varlen is not AttentionBackend.forward_varlen","tryCatchPattern":"try:\n    out = backend.forward_varlen(q, k, v, cu_seqlens=cu, max_seqlen=m, return_softmax_lse=True)\nexcept RuntimeError as e:\n    if \"Unexpected Ascend TND softmax LSE\" in str(e):\n        out = fallback_backend.forward_varlen(q, k, v, cu_seqlens=cu, max_seqlen=m, return_softmax_lse=True)\n    else:\n        raise","preventionTips":["Pin the sgkernel-npu/attentions version validated for your SGLang release","Add a startup smoke test asserting LSE shape on a small batch","Log lse.shape alongside the error for fast diagnosis"],"tags":["ascend","npu","flash-attention","lse","shape-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}