{"record":{"id":"caf06de29a52fc55","repo":"sgl-project/sglang","slug":"rel-bias-sheared-bias-is-only-supported-by-the-f","errorCode":null,"errorMessage":"rel_bias (sheared bias) is only supported by the FA4 backend.","messagePattern":"rel_bias \\(sheared bias\\) is only supported by the FA4 backend\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/flashattention_backend.py","lineNumber":1379,"sourceCode":"        use_cascade_attn = (\n            forward_batch.forward_mode.is_target_verify()\n            and self.topk > 1\n            and not is_swa_layer\n        )\n\n        kwargs = {}\n        if sinks is not None:\n            kwargs[\"sinks\"] = sinks\n        if score_mod is not None:\n            kwargs[\"score_mod\"] = score_mod\n            kwargs[\"aux_tensors\"] = aux_tensors\n        kwargs.update(self._mxfp8_sf_kwargs(layer, forward_batch, q_descale))\n        if fa_k_descale is not None:\n            kwargs[\"k_descale\"] = fa_k_descale\n            kwargs[\"v_descale\"] = fa_v_descale\n        if rel_bias is not None:\n            if self.fa_impl_ver != 4:\n                raise RuntimeError(\n                    \"rel_bias (sheared bias) is only supported by the FA4 backend.\"\n                )\n            if rel_bias_event is not None:\n                # rel_bias (rel_logits) is produced on InklingAttention's alt stream;\n                # join it here -- as late as possible, just before the kernel reads\n                # it -- so rel_logits_proj overlaps the KV-write above.\n                rel_bias_event.wait()\n            kwargs[\"rel_bias\"] = rel_bias\n            if metadata is self.full_cg_prefill_metadata:\n                # Full-CG reuses the cu_seqlens pointer with new values each replay.\n                # Disable its pointer-keyed schedule cache so the graph refreshes it.\n                kwargs[\"rel_bias_prep_cache\"] = None\n            else:\n                if metadata.rel_bias_prep_cache is None:\n                    metadata.rel_bias_prep_cache = {}\n                kwargs[\"rel_bias_prep_cache\"] = metadata.rel_bias_prep_cache\n\n        _fa_out = (","sourceCodeStart":1361,"sourceCodeEnd":1397,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/flashattention_backend.py#L1361-L1397","documentation":"forward_extend raises when rel_bias (sheared relative bias tensors produced on an alt stream by InklingAttention) is non-None on a non-FA4 backend; only FA4 accepts extra bias inputs in its kernel.","triggerScenarios":"Running a model with InklingAttention-style sheared rel_bias while the FA backend version is 2 or 3.","commonSituations":"Loading an experimental relative-bias model on Hopper (FA3) or with FA2 forced; forgetting to pin version 4 for that model.","solutions":["Force the FA4 backend for models that emit rel_bias","Ensure rel_bias/rel_bias_event are only populated on FA4 by gating in the model code","Upgrade SGLang/flash-attn so FA4 is selectable on your GPU"],"exampleFix":"# before\nSGLANG_FLASHATTENTION_VERSION=3 python -m sglang.launch_server --model inkling ...\n# after\nSGLANG_FLASHATTENTION_VERSION=4 python -m sglang.launch_server --model inkling ...","handlingStrategy":"validation","validationCode":"assert rel_bias is None or fa_impl_ver == 4, 'rel_bias requires FA4'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate rel_bias emission on backend version","Document backend requirements per model family"],"tags":["rel-bias","flash-attention-4","extend","sglang"],"backgroundTag":"unsupported-attention-feature","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}