{"record":{"id":"e65c1d665ae1a4e8","repo":"sgl-project/sglang","slug":"score-mod-is-only-supported-by-the-fa4-backend","errorCode":null,"errorMessage":"score_mod is only supported by the FA4 backend.","messagePattern":"score_mod is only supported by the FA4 backend\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/flashattention_backend.py","lineNumber":1249,"sourceCode":"        k: torch.Tensor,\n        v: torch.Tensor,\n        layer: RadixAttention,\n        forward_batch: ForwardBatch,\n        save_kv_cache=True,\n        # For multi-head latent attention\n        q_rope: Optional[torch.Tensor] = None,\n        k_rope: Optional[torch.Tensor] = None,\n        sinks: Optional[torch.Tensor] = None,\n        q_descale: Optional[torch.Tensor] = None,\n        k_descale: Optional[torch.Tensor] = None,\n        v_descale: Optional[torch.Tensor] = None,\n        score_mod=None,\n        aux_tensors=None,\n        rel_bias=None,\n        rel_bias_event=None,\n    ):\n        if score_mod is not None and self.fa_impl_ver != 4:\n            raise RuntimeError(\"score_mod is only supported by the FA4 backend.\")\n        is_cp_mode = (\n            forward_batch.forward_mode.is_context_parallel_extend()\n            and forward_batch.attn_cp_metadata is not None\n            and self.attn_cp_size > 1\n        )\n\n        if k is not None:\n            assert v is not None\n\n            if save_kv_cache and not self.fa_skip_kv_cache:\n                cache_loc = (\n                    forward_batch.out_cache_loc\n                    if not layer.is_cross_attention\n                    else forward_batch.encoder_out_cache_loc\n                )\n                if self.use_mla:\n                    if is_cp_v2_active(forward_batch):\n                        # CP-v2: k/k_rope are rank-local; the strategy gathers","sourceCodeStart":1231,"sourceCodeEnd":1267,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/flashattention_backend.py#L1231-L1267","documentation":"forward_extend raises when a score_mod (flexible-attention score modification, e.g. soft-cap or custom bias function) is requested but the active FA implementation is not FA4, since only FA4 exposes the score_mod hook.","triggerScenarios":"A model layer (e.g. Gemma-style logit softcapping via score_mod or a custom attention variant) passes score_mod while running on FA2/FA3.","commonSituations":"Running a score_mod-dependent model with SGLANG_FLASHATTENTION_VERSION=3 or 2, or on hardware where FA4 is unavailable so startup silently picked FA3.","solutions":["Switch to the FA4 backend (Blackwell GPU + FA4 install, force version 4)","If the model supports it, use its non-score_mod fallback (e.g. separate softcap flag) so score_mod stays None","Use a different attention backend that natively supports the needed modification (e.g. FlashInfer)"],"exampleFix":"# before\nSGLANG_FLASHATTENTION_VERSION=3 python -m sglang.launch_server --model gemma-3-27b ...\n# after\nSGLANG_FLASHATTENTION_VERSION=4 python -m sglang.launch_server --model gemma-3-27b ...","handlingStrategy":"validation","validationCode":"assert score_mod is None or fa_impl_ver == 4, 'score_mod requires FA4 backend'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate score_mod on backend capability in model code","Pin FA4 for softcap/score-mod models in launch scripts"],"tags":["score-mod","flash-attention-4","extend","sglang"],"backgroundTag":"unsupported-attention-feature","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}