{"record":{"id":"682f9d881a821efd","repo":"sgl-project/sglang","slug":"custom-user-provided-score-mod-is-not-supported-on","errorCode":null,"errorMessage":"Custom user-provided score_mod is not supported on SM8x architectures.","messagePattern":"Custom user-provided score_mod is not supported on SM8x architectures\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/interface.py","lineNumber":910,"sourceCode":"        and int(math.ceil(head_dim / 16) * 16) in [128, 192]\n        and int(math.ceil(head_dim_v / 16) * 16) == 128\n        and seqlen_q_packgqa > 2 * tile_m\n        and (tile_m % qhead_per_kvhead == 0 or not pack_gqa)\n        and not qk_blockscaled\n    )\n\n    # hd=256 2CTA forward uses dedicated kernel (Blackwell family)\n    use_dedicated_hd256_kernel = (\n        arch // 10 in [10, 11] and head_dim == 256 and head_dim_v == 256\n    )\n    use_2cta_instrs = use_2cta_instrs or use_dedicated_hd256_kernel\n\n    if softcap is not None:\n        assert score_mod is None, \"softcap and score_mod cannot be used together\"\n        score_mod = utils.create_softcap_scoremod(softcap)\n    elif score_mod is not None:\n        if arch // 10 == 8:\n            raise NotImplementedError(\n                \"Custom user-provided score_mod is not supported on SM8x architectures.\"\n            )\n\n    # hash score and mask mods for compile cache\n    score_mod_hash = utils.hash_callable(score_mod) if score_mod is not None else False\n    mask_mod_hash = utils.hash_callable(mask_mod) if mask_mod is not None else False\n\n    is_varlen = (\n        cu_seqlens_q is not None\n        or cu_seqlens_k is not None\n        or seqused_q is not None\n        or seqused_k is not None\n    )\n\n    # CLC regressed for varlen MHA and dense noncausal. Imbalanced varlen shapes\n    # keep more K/V blocks in flight and hurt L2; dense noncausal mostly just\n    # pays work-stealing overhead.\n    is_varlen_mha = is_varlen and qhead_per_kvhead == 1","sourceCodeStart":892,"sourceCodeEnd":928,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/interface.py#L892-L928","documentation":"Custom user-provided score_mod callables (FlexAttention-style score modification) require kernel features not available on SM8x (Ampere) GPUs; only softcap (converted internally) works there.","triggerScenarios":"Calling _flash_attn_fwd/flash_attn_func with a non-None score_mod on a device whose architecture major is 8 (e.g. A100 is fine at 8.0? no — arch//10==8 covers all SM80/86/89).","commonSituations":"Porting FlexAttention-based masks/score modifications from H100 code to A100/A6000/30xx machines; using capture-based or sigmoid score_mods on Ampere.","solutions":["Drop score_mod on SM8x GPUs; express masking via mask_mod/boolean masks if supported","Replace score_mod with softcap (still supported, since it's converted internally)","Run the workload on SM90+ hardware"],"exampleFix":"// before\nout = fa(q, k, v, score_mod=my_sigmoid_mod)  # on A100\n// after\nout = fa(q, k, v)  # or softcap=50.0 instead of custom score_mod","handlingStrategy":"validation","validationCode":"import torch\nif score_mod is not None and torch.cuda.get_device_capability()[0] == 8 and not isinstance(score_mod, softcap_mod):\n    score_mod = None  # or replace with softcap","typeGuard":"def score_mod_supported(arch: int, score_mod) -> bool:\n    return score_mod is None or arch // 10 != 8","tryCatchPattern":"try:\n    out = fa(q, k, v, score_mod=mod)\nexcept NotImplementedError:\n    out = fa(q, k, v)  # no score_mod on Ampere","preventionTips":["Feature-detect GPU arch before using FlexAttention-style score_mods","Prefer mask_mod or softcap which have wider hardware support"],"tags":["flash-attention","score-mod","flex-attention","sm80","ampere","unsupported-feature"],"backgroundTag":"unsupported-feature-architecture","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}