{"record":{"id":"4e2cdd4d6209ebbd","repo":"sgl-project/sglang","slug":"enable-int8-mamba-checkpoint-only-supports-the-b","errorCode":null,"errorMessage":"--enable-int8-mamba-checkpoint only supports the built-in mamba radix cache; --radix-cache-backend={cfg.radix_cache_backend!r} is not int8-aware. Omit --radix-cache-backend.","messagePattern":"--enable-int8-mamba-checkpoint only supports the built-in mamba radix cache; --radix-cache-backend=(.+?) is not int8-aware\\. Omit --radix-cache-backend\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6818,"sourceCode":"\n    def _handle_int8_mamba_checkpoint(self):\n        # The int8 mamba checkpoint pool is only wired into the built-in\n        # MambaRadixCache. The host-offload path (enabled by\n        # --enable-hierarchical-cache) and custom radix-cache backends are NOT\n        # int8-aware: they would read int8 checkpoint slots as bf16 active slots\n        # (wrong pool / out-of-range). Reject the combination up front rather than\n        # silently corrupting state.\n        cfg = resolving_view(self)\n        if not cfg.enable_int8_mamba_checkpoint:\n            return\n        if cfg.enable_hierarchical_cache:\n            raise ValueError(\n                \"--enable-int8-mamba-checkpoint is not supported together with \"\n                \"--enable-hierarchical-cache: the host-offload path \"\n                \"is not int8-aware. Disable one of them.\"\n            )\n        if cfg.radix_cache_backend is not None:\n            raise ValueError(\n                \"--enable-int8-mamba-checkpoint only supports the built-in mamba \"\n                f\"radix cache; --radix-cache-backend={cfg.radix_cache_backend!r} \"\n                \"is not int8-aware. Omit --radix-cache-backend.\"\n            )\n\n    def _handle_linear_attn_backend(self):\n        cfg = resolving_view(self)\n        import torch\n\n        # SM100+: default to FlashInfer GDN decode (and MTP verify, via pool API)\n        # when the user hasn't explicitly chosen a decode backend and\n        # mamba-ssm-dtype is bf16 (required by FlashInfer GDN on SM100+).\n        # Fixed in FlashInfer v0.6.7: flashinfer-ai/flashinfer#2810\n        if (\n            cfg.linear_attn_decode_backend is None\n            and cfg.linear_attn_backend != \"helion\"\n            and is_sm100_supported()\n            and cfg.mamba_ssm_dtype == \"bfloat16\"","sourceCodeStart":6800,"sourceCodeEnd":6836,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6800-L6836","documentation":"--enable-int8-mamba-checkpoint only works with the built-in Mamba radix cache. Supplying any custom --radix-cache-backend is rejected because third-party/cache backend implementations are not int8-aware and would misinterpret int8 checkpoint slots, corrupting cached state.","triggerScenarios":"Launching with --enable-int8-mamba-checkpoint together with --radix-cache-backend set to any value (e.g. lru, flashinfer, or a custom backend) — the check triggers whenever radix_cache_backend is not None.","commonSituations":"Users who previously selected a custom radix cache backend for memory or eviction reasons and later enable the int8 mamba checkpoint optimization without removing the old flag; configs grown over time where both flags linger.","solutions":["Remove --radix-cache-backend from the launch command so the built-in mamba radix cache is used","Or remove --enable-int8-mamba-checkpoint to keep the custom radix cache backend"],"exampleFix":"# before\n--enable-int8-mamba-checkpoint --radix-cache-backend lru\n# after\n--enable-int8-mamba-checkpoint","handlingStrategy":"validation","validationCode":"if args.enable_int8_mamba_checkpoint:\n    assert args.radix_cache_backend is None, \"int8 mamba checkpoint requires the built-in mamba radix cache\"","typeGuard":"null","tryCatchPattern":null,"preventionTips":["Prune stale flags from launch configs when adding new optimizations","Document which flags each optimization is compatible with in your internal runbooks"],"tags":["sglang","mamba","int8","radix-cache","incompatible-flags","server-args"],"backgroundTag":"incompatible-server-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}