sgl-project/sglang · error · ValueError

--enable-int8-mamba-checkpoint is not supported together wit

Error message

--enable-int8-mamba-checkpoint is not supported together with --enable-hierarchical-cache: the host-offload path is not int8-aware. Disable one of them.

What it means

The int8 Mamba checkpoint pool (enabled by --enable-int8-mamba-checkpoint) is only wired into the built-in MambaRadixCache. The hierarchical-cache host-offload path is not int8-aware and would read int8 checkpoint slots as bf16 active slots, corrupting state, so the combination is rejected up front in _handle_int8_mamba_checkpoint.

Source

Thrown at python/sglang/srt/server_args.py:6812

                    logger.info("Successfully imported FlashInfer mamba module")
                except (ImportError, AttributeError):
                    raise ValueError(flashinfer_error)
            else:
                raise ValueError(flashinfer_error)

    def _handle_int8_mamba_checkpoint(self):
        # The int8 mamba checkpoint pool is only wired into the built-in
        # MambaRadixCache. The host-offload path (enabled by
        # --enable-hierarchical-cache) and custom radix-cache backends are NOT
        # int8-aware: they would read int8 checkpoint slots as bf16 active slots
        # (wrong pool / out-of-range). Reject the combination up front rather than
        # silently corrupting state.
        cfg = resolving_view(self)
        if not cfg.enable_int8_mamba_checkpoint:
            return
        if cfg.enable_hierarchical_cache:
            raise ValueError(
                "--enable-int8-mamba-checkpoint is not supported together with "
                "--enable-hierarchical-cache: the host-offload path "
                "is not int8-aware. Disable one of them."
            )
        if cfg.radix_cache_backend is not None:
            raise ValueError(
                "--enable-int8-mamba-checkpoint only supports the built-in mamba "
                f"radix cache; --radix-cache-backend={cfg.radix_cache_backend!r} "
                "is not int8-aware. Omit --radix-cache-backend."
            )

    def _handle_linear_attn_backend(self):
        cfg = resolving_view(self)
        import torch

        # SM100+: default to FlashInfer GDN decode (and MTP verify, via pool API)
        # when the user hasn't explicitly chosen a decode backend and
        # mamba-ssm-dtype is bf16 (required by FlashInfer GDN on SM100+).

View on GitHub (pinned to 0132848349)

Solutions

1. Drop --enable-hierarchical-cache if int8 mamba checkpoints are required
2. Drop --enable-int8-mamba-checkpoint if hierarchical cache / host offload is required

Example fix

# before
--enable-int8-mamba-checkpoint --enable-hierarchical-cache
# after
--enable-int8-mamba-checkpoint

if args.enable_int8_mamba_checkpoint:
    assert not args.enable_hierarchical_cache, "int8 mamba checkpoint is incompatible with hierarchical cache"

null

When it happens

Trigger: Launching with both --enable-int8-mamba-checkpoint and --enable-hierarchical-cache. Validation runs during ServerArgs resolution and aborts startup before any cache is built.

Common situations: Users combining the int8 checkpoint memory optimization with host offload (HiCache) for large-prefix workloads; config templates that stack all memory-saving flags without checking compatibility matrices.

Related errors

• --enable-int8-mamba-checkpoint only supports the built-in ma
• Mamba storage zero-copy requires page_first layout, got {sel
• --mamba-max-states-per-path must be -1 (unlimited) or a posi
• Stochastic rounding for the Mamba SSM cache requires --mamba
• Stochastic rounding for the Mamba SSM cache is only supporte

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0e7af5ad0316fee2. Report an issue: GitHub.