{"record":{"id":"5baac586b78d3bcb","repo":"sgl-project/sglang","slug":"the-requested-sm120-sheared-bias-specialization-ex","errorCode":null,"errorMessage":"The requested SM120 sheared-bias specialization exceeds shared-memory capacity","messagePattern":"The requested SM120 sheared-bias specialization exceeds shared-memory capacity","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fa4_sm120/runtime.py","lineNumber":981,"sourceCode":"                \"SM120 kernel constraints or shared-memory capacity\"\n            )\n        if has_bias:\n            bias_smem_bytes = (\n                bias_block_size * config.tile_n * (dtype.width // 8) * config.num_stages\n            )\n            total_smem_bytes = (\n                FlashAttentionForwardSm120._smem_usage_in_bytes(\n                    head_dim,\n                    head_dim_v,\n                    config.tile_m,\n                    config.tile_n,\n                    config.num_stages,\n                    False,\n                )\n                + bias_smem_bytes\n            )\n            if total_smem_bytes > utils_basic.get_smem_capacity_in_bytes(\"sm_120\"):\n                raise ValueError(\n                    \"The requested SM120 sheared-bias specialization exceeds \"\n                    \"shared-memory capacity\"\n                )\n        Kernel = FlashAttentionForwardSm120\n        if plan.transpose_qk_pv:\n            from sglang.kernels.ops.attention.fa4_sm120.flash_fwd_decode import (\n                FlashAttentionForwardSm120DecodeTranspose,\n            )\n\n            Kernel = FlashAttentionForwardSm120DecodeTranspose\n        return Kernel(\n            dtype,\n            head_dim,\n            head_dim_v,\n            qhead_per_kvhead,\n            is_causal=is_causal,\n            is_local=is_local,\n            pack_gqa=pack_gqa,","sourceCodeStart":963,"sourceCodeEnd":999,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fa4_sm120/runtime.py#L963-L999","documentation":"When the sheared-bias specialization is active, make_kernel adds bias_smem_bytes (bias block size x tile_n x dtype width x num_stages) on top of the base kernel's shared-memory usage. If the total exceeds the SM120 shared-memory capacity, it raises this ValueError.","triggerScenarios":"Using the sheared-bias path with large bias_block_size, wide bias dtype, many num_stages, or tile_n=128 such that the bias buffer alone blows the smem budget.","commonSituations":"Large bias block sizes for long-context relative bias; configs tuned without bias re-tuned with bias enabled; dtype promotions (fp32 bias) doubling smem needs.","solutions":["Reduce bias_block_size or num_stages in the config","Use a narrower bias dtype if the kernel supports it","Fall back to the non-sheared bias path or a different backend for very large bias footprints"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"from sglang.kernels.ops.attention.fa4_sm120 import utils_basic\ncap = utils_basic.get_smem_capacity_in_bytes(\"sm_120\")\nest = bias_block_size * tile_n * (dtype_bits // 8) * num_stages\nif est > cap * 0.8:  # leave room for base kernel smem\n    num_stages = 2  # or shrink bias_block_size","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Budget smem for the bias buffer before enabling the sheared-bias specialization","Retry with fewer num_stages on this specific ValueError"],"tags":["flash-attention","sm120","shared-memory","relative-bias"],"backgroundTag":"kernel-resource-limit-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}