sgl-project/sglang · error · ValueError

--enable-int8-mamba-checkpoint only supports the built-in ma

Error message

--enable-int8-mamba-checkpoint only supports the built-in mamba radix cache; --radix-cache-backend={cfg.radix_cache_backend!r} is not int8-aware. Omit --radix-cache-backend.

What it means

--enable-int8-mamba-checkpoint only works with the built-in Mamba radix cache. Supplying any custom --radix-cache-backend is rejected because third-party/cache backend implementations are not int8-aware and would misinterpret int8 checkpoint slots, corrupting cached state.

Source

Thrown at python/sglang/srt/server_args.py:6818

    def _handle_int8_mamba_checkpoint(self):
        # The int8 mamba checkpoint pool is only wired into the built-in
        # MambaRadixCache. The host-offload path (enabled by
        # --enable-hierarchical-cache) and custom radix-cache backends are NOT
        # int8-aware: they would read int8 checkpoint slots as bf16 active slots
        # (wrong pool / out-of-range). Reject the combination up front rather than
        # silently corrupting state.
        cfg = resolving_view(self)
        if not cfg.enable_int8_mamba_checkpoint:
            return
        if cfg.enable_hierarchical_cache:
            raise ValueError(
                "--enable-int8-mamba-checkpoint is not supported together with "
                "--enable-hierarchical-cache: the host-offload path "
                "is not int8-aware. Disable one of them."
            )
        if cfg.radix_cache_backend is not None:
            raise ValueError(
                "--enable-int8-mamba-checkpoint only supports the built-in mamba "
                f"radix cache; --radix-cache-backend={cfg.radix_cache_backend!r} "
                "is not int8-aware. Omit --radix-cache-backend."
            )

    def _handle_linear_attn_backend(self):
        cfg = resolving_view(self)
        import torch

        # SM100+: default to FlashInfer GDN decode (and MTP verify, via pool API)
        # when the user hasn't explicitly chosen a decode backend and
        # mamba-ssm-dtype is bf16 (required by FlashInfer GDN on SM100+).
        # Fixed in FlashInfer v0.6.7: flashinfer-ai/flashinfer#2810
        if (
            cfg.linear_attn_decode_backend is None
            and cfg.linear_attn_backend != "helion"
            and is_sm100_supported()
            and cfg.mamba_ssm_dtype == "bfloat16"

View on GitHub (pinned to 0132848349)

Solutions

Remove --radix-cache-backend from the launch command so the built-in mamba radix cache is used
Or remove --enable-int8-mamba-checkpoint to keep the custom radix cache backend

Example fix

# before
--enable-int8-mamba-checkpoint --radix-cache-backend lru
# after
--enable-int8-mamba-checkpoint

Defensive patterns

Strategy: validation

Validate before calling

if args.enable_int8_mamba_checkpoint:
    assert args.radix_cache_backend is None, "int8 mamba checkpoint requires the built-in mamba radix cache"

Type guard

null

Prevention

Prune stale flags from launch configs when adding new optimizations
Document which flags each optimization is compatible with in your internal runbooks

When it happens

Trigger: Launching with --enable-int8-mamba-checkpoint together with --radix-cache-backend set to any value (e.g. lru, flashinfer, or a custom backend) — the check triggers whenever radix_cache_backend is not None.

Common situations: Users who previously selected a custom radix cache backend for memory or eviction reasons and later enable the int8 mamba checkpoint optimization without removing the old flag; configs grown over time where both flags linger.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4e2cdd4d6209ebbd. Report an issue: GitHub.