{"record":{"id":"869ae8d7316b178f","repo":"sgl-project/sglang","slug":"head-dim-head-dim-v-head-dim-head-dim-v","errorCode":null,"errorMessage":"(head_dim, head_dim_v)=({head_dim}, {head_dim_v}) exceeds SM120 shared-memory capacity ({smem_capacity} bytes)","messagePattern":"\\(head_dim, head_dim_v\\)=\\((.+?), (.+?)\\) exceeds SM120 shared-memory capacity \\((.+?) bytes\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fa4_sm120/flash_fwd.py","lineNumber":686,"sourceCode":"            # packed GQA local attention reaches its steady state.  Keep MHA\n            # and larger SM arrays on the cross-SKU HD256 LPT calibration.\n            preferred = (64, 48, 1)\n            candidates = (preferred,) + tuple(\n                candidate for candidate in candidates if candidate != preferred\n            )\n        for tile_m, tile_n, num_stages in candidates:\n            if (\n                FlashAttentionForwardSm120._smem_usage_in_bytes(\n                    head_dim,\n                    head_dim_v,\n                    tile_m,\n                    tile_n,\n                    num_stages,\n                )\n                <= smem_capacity\n            ):\n                return tile_m, tile_n\n        raise ValueError(\n            f\"(head_dim, head_dim_v)=({head_dim}, {head_dim_v}) exceeds \"\n            f\"SM120 shared-memory capacity ({smem_capacity} bytes)\"\n        )\n\n    @staticmethod\n    def get_fwd_num_stages(\n        head_dim: int, head_dim_v: int, tile_m: int, tile_n: int\n    ) -> int:\n        \"\"\"Return the public pipeline specialization depth.\"\"\"\n        return 1\n\n    @staticmethod\n    def get_fwd_num_threads(\n        head_dim: int,\n        head_dim_v: int,\n        tile_m: int,\n        tile_n: int,\n        paged_kv: bool = False,","sourceCodeStart":668,"sourceCodeEnd":704,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fa4_sm120/flash_fwd.py#L668-L704","documentation":"get_fwd_tile_size searches tile shapes for the SM120 FlashAttention forward kernel such that the pipeline's shared-memory usage fits the GPU's smem capacity. For very large head_dim/head_dim_v combinations no tile fits, so it raises this ValueError.","triggerScenarios":"Requesting a forward tile size with head_dim + head_dim_v so large (e.g. 256+256) that even the smallest tile's smem footprint exceeds the SM120 (Blackwell) shared-memory capacity.","commonSituations":"Emerging model architectures with head_dim > 128/256; overriding head_dim_v in a custom model; running a config tuned for a datacenter GPU (H100, larger smem) on a consumer Blackwell card with less shared memory.","solutions":["Reduce head_dim or head_dim_v (e.g. cap at 128/256) for the SM120 path","Route such large-head-dim models to a different attention backend that supports them","Query utils to check smem capacity and pre-validate the config instead of crashing at tile search"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from sglang.kernels.ops.attention.fa4_sm120 import utils_basic\ncap = utils_basic.get_smem_capacity_in_bytes(\"sm_120\")\n# pre-check: rough smem need grows ~ (head_dim + head_dim_v); reject > 128+128 configs early","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Cap head_dim/head_dim_v at 128 for the SM120 FA4 path in model routing","Do not port H100-tuned large-head-dim configs to consumer Blackwell"],"tags":["flash-attention","sm120","shared-memory","head-dim","blackwell"],"backgroundTag":"kernel-resource-limit-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}