{"record":{"id":"0e7af5ad0316fee2","repo":"sgl-project/sglang","slug":"enable-int8-mamba-checkpoint-is-not-supported-to","errorCode":null,"errorMessage":"--enable-int8-mamba-checkpoint is not supported together with --enable-hierarchical-cache: the host-offload path is not int8-aware. Disable one of them.","messagePattern":"--enable-int8-mamba-checkpoint is not supported together with --enable-hierarchical-cache: the host-offload path is not int8-aware\\. Disable one of them\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6812,"sourceCode":"\n                    logger.info(\"Successfully imported FlashInfer mamba module\")\n                except (ImportError, AttributeError):\n                    raise ValueError(flashinfer_error)\n            else:\n                raise ValueError(flashinfer_error)\n\n    def _handle_int8_mamba_checkpoint(self):\n        # The int8 mamba checkpoint pool is only wired into the built-in\n        # MambaRadixCache. The host-offload path (enabled by\n        # --enable-hierarchical-cache) and custom radix-cache backends are NOT\n        # int8-aware: they would read int8 checkpoint slots as bf16 active slots\n        # (wrong pool / out-of-range). Reject the combination up front rather than\n        # silently corrupting state.\n        cfg = resolving_view(self)\n        if not cfg.enable_int8_mamba_checkpoint:\n            return\n        if cfg.enable_hierarchical_cache:\n            raise ValueError(\n                \"--enable-int8-mamba-checkpoint is not supported together with \"\n                \"--enable-hierarchical-cache: the host-offload path \"\n                \"is not int8-aware. Disable one of them.\"\n            )\n        if cfg.radix_cache_backend is not None:\n            raise ValueError(\n                \"--enable-int8-mamba-checkpoint only supports the built-in mamba \"\n                f\"radix cache; --radix-cache-backend={cfg.radix_cache_backend!r} \"\n                \"is not int8-aware. Omit --radix-cache-backend.\"\n            )\n\n    def _handle_linear_attn_backend(self):\n        cfg = resolving_view(self)\n        import torch\n\n        # SM100+: default to FlashInfer GDN decode (and MTP verify, via pool API)\n        # when the user hasn't explicitly chosen a decode backend and\n        # mamba-ssm-dtype is bf16 (required by FlashInfer GDN on SM100+).","sourceCodeStart":6794,"sourceCodeEnd":6830,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6794-L6830","documentation":"The int8 Mamba checkpoint pool (enabled by --enable-int8-mamba-checkpoint) is only wired into the built-in MambaRadixCache. The hierarchical-cache host-offload path is not int8-aware and would read int8 checkpoint slots as bf16 active slots, corrupting state, so the combination is rejected up front in _handle_int8_mamba_checkpoint.","triggerScenarios":"Launching with both --enable-int8-mamba-checkpoint and --enable-hierarchical-cache. Validation runs during ServerArgs resolution and aborts startup before any cache is built.","commonSituations":"Users combining the int8 checkpoint memory optimization with host offload (HiCache) for large-prefix workloads; config templates that stack all memory-saving flags without checking compatibility matrices.","solutions":["Drop --enable-hierarchical-cache if int8 mamba checkpoints are required","Drop --enable-int8-mamba-checkpoint if hierarchical cache / host offload is required"],"exampleFix":"# before\n--enable-int8-mamba-checkpoint --enable-hierarchical-cache\n# after\n--enable-int8-mamba-checkpoint","handlingStrategy":"validation","validationCode":"if args.enable_int8_mamba_checkpoint:\n    assert not args.enable_hierarchical_cache, \"int8 mamba checkpoint is incompatible with hierarchical cache\"","typeGuard":"null","tryCatchPattern":null,"preventionTips":["Maintain a known-incompatible flag matrix and lint launch configs against it","Enable memory optimizations one at a time, verifying startup after each"],"tags":["sglang","mamba","int8","hierarchical-cache","incompatible-flags","server-args"],"backgroundTag":"incompatible-server-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}