sgl-project/sglang · error · ValueError

{flashinfer_error}

Error message

{flashinfer_error}

What it means

Raised when --mamba-backend flashinfer is set, FlashInfer is installed, but `import flashinfer.mamba` raises ImportError/AttributeError. The flashinfer pip package in the environment predates or lacks the mamba submodule that sglang's FlashInfer Mamba path needs.

Source

Thrown at python/sglang/srt/server_args.py:6797

                )

        if cfg.mamba_backend == "flashinfer":
            flashinfer_error = (
                "FlashInfer mamba module not available, please check the "
                "FlashInfer installation."
            )
            if cfg.enable_mamba_cache_stochastic_rounding:
                flashinfer_error += (
                    " Stochastic rounding with --mamba-backend flashinfer "
                    "requires FlashInfer Mamba and --mamba-ssm-dtype float16."
                )
            if is_flashinfer_available():
                try:
                    import flashinfer.mamba  # noqa: F401

                    logger.info("Successfully imported FlashInfer mamba module")
                except (ImportError, AttributeError):
                    raise ValueError(flashinfer_error)
            else:
                raise ValueError(flashinfer_error)

    def _handle_int8_mamba_checkpoint(self):
        # The int8 mamba checkpoint pool is only wired into the built-in
        # MambaRadixCache. The host-offload path (enabled by
        # --enable-hierarchical-cache) and custom radix-cache backends are NOT
        # int8-aware: they would read int8 checkpoint slots as bf16 active slots
        # (wrong pool / out-of-range). Reject the combination up front rather than
        # silently corrupting state.
        cfg = resolving_view(self)
        if not cfg.enable_int8_mamba_checkpoint:
            return
        if cfg.enable_hierarchical_cache:
            raise ValueError(
                "--enable-int8-mamba-checkpoint is not supported together with "
                "--enable-hierarchical-cache: the host-offload path "
                "is not int8-aware. Disable one of them."

View on GitHub (pinned to 0132848349)

Solutions

1. Upgrade flashinfer to a release that includes the mamba module (e.g. pip install -U flashinfer-python) matching your CUDA version
2. Reinstall flashinfer with the correct CUDA variant for your driver/toolkit
3. Fall back to --mamba-backend triton until the environment is fixed
4. If the flag --enable-mamba-cache-stochastic-rounding is also set, ensure --mamba-ssm-dtype float16 as the appended message requires

Example fix

# before
pip install flashinfer-python==0.1.x  # no mamba module
python -m sglang.launch_server --mamba-backend flashinfer ...
# after
pip install -U flashinfer-python
python -m sglang.launch_server --mamba-backend flashinfer ...

def flashinfer_mamba_ok() -> bool:
    try:
        import flashinfer.mamba  # noqa: F401
        return True
    except (ImportError, AttributeError):
        return False
assert flashinfer_mamba_ok(), "upgrade flashinfer to a build with the mamba module"

null

When it happens

Trigger: Launching with --mamba-backend flashinfer on a machine whose flashinfer version does not ship flashinfer.mamba (older wheel, CPU-only wheel, or partial install). The import is attempted inside _handle_mamba_backend and any ImportError/AttributeError is converted to this ValueError.

Common situations: Pinned or stale flashinfer version in the image; a flashinfer wheel built for a different CUDA version that fails module import; environment drift after upgrading sglang which now requires the mamba submodule.

Related errors

• Please install mooncake by following the instructions at htt
• gRPC mode requires the smg-grpc-servicer package. If not ins
• ModelOpt is not available. Please install modelopt.
• The MLX tensor bridge requires MLX >= 0.32.0
• Failed to load serve backend {name!r} from {self._entry_poin

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/657237f826f98df5. Report an issue: GitHub.