{"record":{"id":"657237f826f98df5","repo":"sgl-project/sglang","slug":"flashinfer-error","errorCode":null,"errorMessage":"{flashinfer_error}","messagePattern":"\\{flashinfer_error\\}","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6797,"sourceCode":"                )\n\n        if cfg.mamba_backend == \"flashinfer\":\n            flashinfer_error = (\n                \"FlashInfer mamba module not available, please check the \"\n                \"FlashInfer installation.\"\n            )\n            if cfg.enable_mamba_cache_stochastic_rounding:\n                flashinfer_error += (\n                    \" Stochastic rounding with --mamba-backend flashinfer \"\n                    \"requires FlashInfer Mamba and --mamba-ssm-dtype float16.\"\n                )\n            if is_flashinfer_available():\n                try:\n                    import flashinfer.mamba  # noqa: F401\n\n                    logger.info(\"Successfully imported FlashInfer mamba module\")\n                except (ImportError, AttributeError):\n                    raise ValueError(flashinfer_error)\n            else:\n                raise ValueError(flashinfer_error)\n\n    def _handle_int8_mamba_checkpoint(self):\n        # The int8 mamba checkpoint pool is only wired into the built-in\n        # MambaRadixCache. The host-offload path (enabled by\n        # --enable-hierarchical-cache) and custom radix-cache backends are NOT\n        # int8-aware: they would read int8 checkpoint slots as bf16 active slots\n        # (wrong pool / out-of-range). Reject the combination up front rather than\n        # silently corrupting state.\n        cfg = resolving_view(self)\n        if not cfg.enable_int8_mamba_checkpoint:\n            return\n        if cfg.enable_hierarchical_cache:\n            raise ValueError(\n                \"--enable-int8-mamba-checkpoint is not supported together with \"\n                \"--enable-hierarchical-cache: the host-offload path \"\n                \"is not int8-aware. Disable one of them.\"","sourceCodeStart":6779,"sourceCodeEnd":6815,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6779-L6815","documentation":"Raised when --mamba-backend flashinfer is set, FlashInfer is installed, but `import flashinfer.mamba` raises ImportError/AttributeError. The flashinfer pip package in the environment predates or lacks the mamba submodule that sglang's FlashInfer Mamba path needs.","triggerScenarios":"Launching with --mamba-backend flashinfer on a machine whose flashinfer version does not ship flashinfer.mamba (older wheel, CPU-only wheel, or partial install). The import is attempted inside _handle_mamba_backend and any ImportError/AttributeError is converted to this ValueError.","commonSituations":"Pinned or stale flashinfer version in the image; a flashinfer wheel built for a different CUDA version that fails module import; environment drift after upgrading sglang which now requires the mamba submodule.","solutions":["Upgrade flashinfer to a release that includes the mamba module (e.g. pip install -U flashinfer-python) matching your CUDA version","Reinstall flashinfer with the correct CUDA variant for your driver/toolkit","Fall back to --mamba-backend triton until the environment is fixed","If the flag --enable-mamba-cache-stochastic-rounding is also set, ensure --mamba-ssm-dtype float16 as the appended message requires"],"exampleFix":"# before\npip install flashinfer-python==0.1.x  # no mamba module\npython -m sglang.launch_server --mamba-backend flashinfer ...\n# after\npip install -U flashinfer-python\npython -m sglang.launch_server --mamba-backend flashinfer ...","handlingStrategy":"validation","validationCode":"def flashinfer_mamba_ok() -> bool:\n    try:\n        import flashinfer.mamba  # noqa: F401\n        return True\n    except (ImportError, AttributeError):\n        return False\nassert flashinfer_mamba_ok(), \"upgrade flashinfer to a build with the mamba module\"","typeGuard":"null","tryCatchPattern":null,"preventionTips":["Pin a flashinfer version known to ship flashinfer.mamba","Add a smoke-test step (import flashinfer.mamba) to container build/CI"],"tags":["sglang","mamba","flashinfer","dependency","import-error"],"backgroundTag":"missing-optional-dependency","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}