sgl-project/sglang · error · ValueError

Currently only {RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACK

Error message

Currently only {RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND} attention backends are supported for deterministic inference with absorbed-MLA models. But you're using {attention_backend}.

What it means

Raised when deterministic inference is enabled with an absorbed-MLA (DeepSeek-family) model but the chosen attention backend is not in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND. Deterministic replay of absorbed MLA is only implemented for specific radix-capable backends, so others cannot guarantee bit-identical attention results.

Source

Thrown at python/sglang/srt/server_args.py:9234

                        "DeepseekV32ForCausalLM",
                        "MistralLarge3ForCausalLM",
                        "PixtralForConditionalGeneration",
                        "GlmMoeDsaForCausalLM",
                        "Glm4MoeLiteForCausalLM",
                    ]
                except Exception:
                    pass

            # Check attention backend
            run_post_process_pass(self, _deterministic_attention_backend)

            attention_backend = resolved_view(self).attention_backend
            if is_deepseek_model:
                if (
                    attention_backend
                    not in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND
                ):
                    raise ValueError(
                        f"Currently only {RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND} attention backends are supported for deterministic inference with absorbed-MLA models. But you're using {attention_backend}."
                    )
                if attention_backend == "fa4" and not is_sm100_or_sm110_supported():
                    raise ValueError(
                        "Deterministic inference with absorbed-MLA models on the fa4 "
                        "attention backend requires SM100/SM110: it runs "
                        "absorbed MLA, whose qv argument flash_attn.cute only "
                        "implements on those archs."
                    )

            if attention_backend not in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND:
                # Currently, only certain backends support radix cache. Support for other backends is in progress
                self._declare(
                    "_handle_deterministic_inference",
                    disable_radix_cache=True,
                )
                logger.warning(
                    f"Currently radix cache is not compatible with {attention_backend} attention backend for deterministic inference. It will be supported in the future."

View on GitHub (pinned to 0132848349)

Solutions

  1. Switch to one of the backends in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND (inspect its value in python/sglang/srt/server_args.py)
  2. Or disable deterministic inference / rl_on_policy_target if reproducibility is not required
  3. For DeepSeek models prefer the supported backend pair listed in the constant

Example fix

# before
--attention-backend flashinfer --rl-on-policy-target ...
# after
--attention-backend fa3 --rl-on-policy-target ...
Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.server_args import RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND
if deterministic and is_deepseek:
    assert args.attention_backend in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND, (
        f"need one of {RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND}")

Type guard

def backend_supports_deterministic_mla(backend: str) -> bool:
    from sglang.srt.server_args import RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND
    return backend in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND

Try / catch

try:
    ServerArgs(**cli)
except ValueError as e:
    if 'deterministic inference' in str(e):
        cli['attention_backend'] = 'fa3'  # fallback to a supported backend
        ServerArgs(**cli)
    else:
        raise

Prevention

When it happens

Trigger: Enabling deterministic inference (e.g. via rl_on_policy_target or explicit flag) with a DeepSeek/absorbed-MLA model while attention_backend is something other than the supported set (e.g. flashinfer or triton variants).

Common situations: Switching attention backends for perf tuning while keeping deterministic inference on; upgrading SGLang where the supported backend list changed; RL on-policy workflows hitting an unsupported default backend.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/827d01d8cdd14a5d. Report an issue: GitHub.