{"record":{"id":"7e6d4c2c1df8f45e","repo":"sgl-project/sglang","slug":"sm120-relative-bias-currently-supports-head-dim-an","errorCode":null,"errorMessage":"SM120 relative bias currently supports head_dim and head_dim_v up to 128","messagePattern":"SM120 relative bias currently supports head_dim and head_dim_v up to 128","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fa4_sm120/runtime.py","lineNumber":627,"sourceCode":"        tile_mn: Optional[tuple[int, int]],\n        has_bias: bool,\n        total_q_rows: int,\n        num_sms: Optional[int],\n        num_batch: int,\n        seqlen_q: Optional[int],\n        seqlen_k: Optional[int],\n        num_head_kv: int,\n        qhead_per_kvhead: int,\n        is_causal: bool,\n        is_local: bool,\n        window_size_left: Optional[int],\n        window_size_right: Optional[int],\n        pack_gqa: bool,\n        paged_kv: bool,\n    ) -> Sm120ForwardConfig:\n        if has_bias:\n            if max(head_dim, head_dim_v) > 128:\n                raise ValueError(\n                    \"SM120 relative bias currently supports head_dim and \"\n                    \"head_dim_v up to 128\"\n                )\n            if tile_mn is not None and tile_mn != (64, 128):\n                raise ValueError(\"SM120 relative bias requires tile_mn=(64, 128)\")\n            tile_m, tile_n = 64, 128\n        elif tile_mn is None:\n            tile_m, tile_n = FlashAttentionForwardSm120.get_fwd_tile_size(\n                head_dim,\n                head_dim_v,\n                total_q_rows=total_q_rows,\n                num_sms=num_sms,\n                num_batch=num_batch,\n                seqlen_q=seqlen_q,\n                seqlen_k=seqlen_k,\n                num_head_kv=num_head_kv,\n                qhead_per_kvhead=qhead_per_kvhead,\n                is_causal=is_causal,","sourceCodeStart":609,"sourceCodeEnd":645,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fa4_sm120/runtime.py#L609-L645","documentation":"The SM120 FlashAttention relative-position-bias specialization is only implemented for head_dim and head_dim_v up to 128. When has_bias=True and either dim exceeds 128, select_config raises before any kernel is built.","triggerScenarios":"Calling _flash_attn_fwd with has_bias=True and max(head_dim, head_dim_v) > 128.","commonSituations":"Enabling ALiBi/relative bias on a model with head_dim 256 (e.g. some long-context or audio models); porting a bias-enabled workload from another FA backend that allowed it.","solutions":["Use a backend without the 128-dim bias restriction for head_dim > 128 models","If possible, restructure the model (e.g. split heads) to bring head_dim_v <= 128","Drop the bias or switch to an additive-bias formulation supported at larger dims"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if has_bias:\n    assert max(head_dim, head_dim_v) <= 128, \"SM120 bias path supports dims <= 128\"","typeGuard":"def bias_dims_ok(head_dim: int, head_dim_v: int) -> bool:\n    return max(head_dim, head_dim_v) <= 128","tryCatchPattern":null,"preventionTips":["Route bias-enabled large-head-dim models to another backend at init, not at forward time"],"tags":["flash-attention","sm120","relative-bias","head-dim"],"backgroundTag":"unsupported-argument-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}