sgl-project/sglang · error · RuntimeError

rel_bias (sheared bias) is only supported by the FA4 backend

Error message

rel_bias (sheared bias) is only supported by the FA4 backend.

What it means

forward_extend raises when rel_bias (sheared relative bias tensors produced on an alt stream by InklingAttention) is non-None on a non-FA4 backend; only FA4 accepts extra bias inputs in its kernel.

Source

Thrown at python/sglang/srt/layers/attention/flashattention_backend.py:1379

        use_cascade_attn = (
            forward_batch.forward_mode.is_target_verify()
            and self.topk > 1
            and not is_swa_layer
        )

        kwargs = {}
        if sinks is not None:
            kwargs["sinks"] = sinks
        if score_mod is not None:
            kwargs["score_mod"] = score_mod
            kwargs["aux_tensors"] = aux_tensors
        kwargs.update(self._mxfp8_sf_kwargs(layer, forward_batch, q_descale))
        if fa_k_descale is not None:
            kwargs["k_descale"] = fa_k_descale
            kwargs["v_descale"] = fa_v_descale
        if rel_bias is not None:
            if self.fa_impl_ver != 4:
                raise RuntimeError(
                    "rel_bias (sheared bias) is only supported by the FA4 backend."
                )
            if rel_bias_event is not None:
                # rel_bias (rel_logits) is produced on InklingAttention's alt stream;
                # join it here -- as late as possible, just before the kernel reads
                # it -- so rel_logits_proj overlaps the KV-write above.
                rel_bias_event.wait()
            kwargs["rel_bias"] = rel_bias
            if metadata is self.full_cg_prefill_metadata:
                # Full-CG reuses the cu_seqlens pointer with new values each replay.
                # Disable its pointer-keyed schedule cache so the graph refreshes it.
                kwargs["rel_bias_prep_cache"] = None
            else:
                if metadata.rel_bias_prep_cache is None:
                    metadata.rel_bias_prep_cache = {}
                kwargs["rel_bias_prep_cache"] = metadata.rel_bias_prep_cache

        _fa_out = (

View on GitHub (pinned to 0132848349)

Solutions

1. Force the FA4 backend for models that emit rel_bias
2. Ensure rel_bias/rel_bias_event are only populated on FA4 by gating in the model code
3. Upgrade SGLang/flash-attn so FA4 is selectable on your GPU

Example fix

# before
SGLANG_FLASHATTENTION_VERSION=3 python -m sglang.launch_server --model inkling ...
# after
SGLANG_FLASHATTENTION_VERSION=4 python -m sglang.launch_server --model inkling ...

assert rel_bias is None or fa_impl_ver == 4, 'rel_bias requires FA4'

When it happens

Trigger: Running a model with InklingAttention-style sheared rel_bias while the FA backend version is 2 or 3.

Common situations: Loading an experimental relative-bias model on Hopper (FA3) or with FA2 forced; forgetting to pin version 4 for that model.

Related errors

• score_mod is only supported by the FA4 backend.
• MXFP8 KV cache requires the FA4 backend.
• MXFP8 KV cache requires per-token Q scales (q_descale) from
• VisionFlash4Attention is only available for cuda
• Unknown serve backend {name!r}. Available values: {available

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/caf06de29a52fc55. Report an issue: GitHub.