{"record":{"id":"4e60d1276552a344","repo":"sgl-project/sglang","slug":"stochastic-rounding-for-the-mamba-ssm-cache-requir-4e60d1","errorCode":null,"errorMessage":"Stochastic rounding for the Mamba SSM cache requires --mamba-ssm-dtype float16, got {cfg.mamba_ssm_dtype!r}. Run with --mamba-ssm-dtype float16 or disable --enable-mamba-cache-stochastic-rounding.","messagePattern":"Stochastic rounding for the Mamba SSM cache requires --mamba-ssm-dtype float16, got (.+?)\\. Run with --mamba-ssm-dtype float16 or disable --enable-mamba-cache-stochastic-rounding\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6759,"sourceCode":"    def _handle_grammar_backend(self):\n        cfg = resolving_view(self)\n        if cfg.grammar_backend is None:\n            self._declare(\"_handle_grammar_backend\", grammar_backend=\"xgrammar\")\n\n    def _handle_mamba_backend(self):\n        cfg = resolving_view(self)\n        if cfg.mamba_cache_philox_rounds < 0:\n            raise ValueError(\"--mamba-cache-philox-rounds must be non-negative.\")\n\n        if cfg.mamba_max_states_per_path == 0 or cfg.mamba_max_states_per_path < -1:\n            raise ValueError(\n                \"--mamba-max-states-per-path must be -1 (unlimited) or a positive \"\n                f\"integer, got {cfg.mamba_max_states_per_path}.\"\n            )\n\n        if cfg.enable_mamba_cache_stochastic_rounding:\n            if cfg.mamba_ssm_dtype != \"float16\":\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache requires \"\n                    f\"--mamba-ssm-dtype float16, got {cfg.mamba_ssm_dtype!r}. \"\n                    \"Run with --mamba-ssm-dtype float16 or disable \"\n                    \"--enable-mamba-cache-stochastic-rounding.\"\n                )\n            if not is_cuda():\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache is only \"\n                    \"supported on NVIDIA CUDA platforms. Disable \"\n                    \"--enable-mamba-cache-stochastic-rounding on this platform.\"\n                )\n            if cfg.mamba_backend == \"triton\" and not is_sm100_supported():\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache with \"\n                    \"--mamba-backend triton requires SM100 with CUDA >= 12.8 \"\n                    \"because it uses the cvt.rs.f16x2.f32 PTX instruction. On \"\n                    \"H100/SM90, run with --mamba-backend flashinfer \"\n                    \"--mamba-ssm-dtype float16, or disable \"","sourceCodeStart":6741,"sourceCodeEnd":6777,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6741-L6777","documentation":"Stochastic rounding for the Mamba SSM cache (enabled via --enable-mamba-cache-stochastic-rounding) is only implemented for the float16 SSM state dtype. The kernel that performs stochastic rounding operates on fp16 state, so any other --mamba-ssm-dtype (e.g. bfloat16, float32) is rejected up front.","triggerScenarios":"Starting the server with --enable-mamba-cache-stochastic-rounding while --mamba-ssm-dtype is anything other than float16. Checked inside _handle_mamba_backend during ServerArgs resolution, so it aborts startup immediately.","commonSituations":"Users copying a bfloat16 SSM config (common for quality reasons) and layering the stochastic-rounding optimization on top; or defaults changing between versions so mamba_ssm_dtype is no longer float16 when the rounding flag is set.","solutions":["Add --mamba-ssm-dtype float16 to the launch command","Disable stochastic rounding by dropping --enable-mamba-cache-stochastic-rounding"],"exampleFix":"# before\n--enable-mamba-cache-stochastic-rounding --mamba-ssm-dtype bfloat16\n# after\n--enable-mamba-cache-stochastic-rounding --mamba-ssm-dtype float16","handlingStrategy":"validation","validationCode":"sr = args.enable_mamba_cache_stochastic_rounding\nif sr:\n    assert args.mamba_ssm_dtype == \"float16\", \"stochastic rounding requires --mamba-ssm-dtype float16\"","typeGuard":"null","tryCatchPattern":null,"preventionTips":["Keep a compatibility table of flag pairs; stochastic rounding is coupled to float16 state","Set both flags together in one config macro instead of independently"],"tags":["sglang","mamba","stochastic-rounding","dtype","server-args"],"backgroundTag":"incompatible-dtype-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}