{"record":{"id":"b72816f1165dd2d3","repo":"sgl-project/sglang","slug":"stochastic-rounding-for-the-mamba-ssm-cache-is-onl","errorCode":null,"errorMessage":"Stochastic rounding for the Mamba SSM cache is only supported on NVIDIA CUDA platforms. Disable --enable-mamba-cache-stochastic-rounding on this platform.","messagePattern":"Stochastic rounding for the Mamba SSM cache is only supported on NVIDIA CUDA platforms\\. Disable --enable-mamba-cache-stochastic-rounding on this platform\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6766,"sourceCode":"        if cfg.mamba_cache_philox_rounds < 0:\n            raise ValueError(\"--mamba-cache-philox-rounds must be non-negative.\")\n\n        if cfg.mamba_max_states_per_path == 0 or cfg.mamba_max_states_per_path < -1:\n            raise ValueError(\n                \"--mamba-max-states-per-path must be -1 (unlimited) or a positive \"\n                f\"integer, got {cfg.mamba_max_states_per_path}.\"\n            )\n\n        if cfg.enable_mamba_cache_stochastic_rounding:\n            if cfg.mamba_ssm_dtype != \"float16\":\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache requires \"\n                    f\"--mamba-ssm-dtype float16, got {cfg.mamba_ssm_dtype!r}. \"\n                    \"Run with --mamba-ssm-dtype float16 or disable \"\n                    \"--enable-mamba-cache-stochastic-rounding.\"\n                )\n            if not is_cuda():\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache is only \"\n                    \"supported on NVIDIA CUDA platforms. Disable \"\n                    \"--enable-mamba-cache-stochastic-rounding on this platform.\"\n                )\n            if cfg.mamba_backend == \"triton\" and not is_sm100_supported():\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache with \"\n                    \"--mamba-backend triton requires SM100 with CUDA >= 12.8 \"\n                    \"because it uses the cvt.rs.f16x2.f32 PTX instruction. On \"\n                    \"H100/SM90, run with --mamba-backend flashinfer \"\n                    \"--mamba-ssm-dtype float16, or disable \"\n                    \"--enable-mamba-cache-stochastic-rounding.\"\n                )\n\n        if cfg.mamba_backend == \"flashinfer\":\n            flashinfer_error = (\n                \"FlashInfer mamba module not available, please check the \"\n                \"FlashInfer installation.\"","sourceCodeStart":6748,"sourceCodeEnd":6784,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6748-L6784","documentation":"The stochastic-rounding Mamba SSM cache kernels are CUDA-only; the is_cuda() platform check in _handle_mamba_backend fails on ROCm, CPU, or any non-NVIDIA build, and the flag combination is rejected at startup rather than crashing later inside a kernel launch.","triggerScenarios":"Launching with --enable-mamba-cache-stochastic-rounding on a machine where is_cuda() returns False (AMD ROCm, CPU-only node, non-CUDA build of sglang).","commonSituations":"Running the same launch script on an AMD MI300x or CPU-only dev box that worked on an NVIDIA cluster; CI jobs that don't gate the flag on platform.","solutions":["Remove --enable-mamba-cache-stochastic-rounding from the launch command on this platform","Run on an NVIDIA CUDA machine if stochastic rounding is required","Gate the flag in your launcher: only add it when torch.cuda.is_available() and the device is NVIDIA"],"exampleFix":"# before\npython -m sglang.launch_server --enable-mamba-cache-stochastic-rounding ...  # on ROCm\n# after\npython -m sglang.launch_server ...  # flag omitted on non-CUDA platforms","handlingStrategy":"validation","validationCode":"import torch\nif args.enable_mamba_cache_stochastic_rounding:\n    assert torch.cuda.is_available(), \"stochastic rounding requires NVIDIA CUDA\"","typeGuard":"null","tryCatchPattern":null,"preventionTips":["Gate CUDA-only flags behind a torch.cuda.is_available() check in your launcher","Maintain per-platform launch profiles (cuda vs rocm) rather than one shared script"],"tags":["sglang","mamba","cuda-only","platform","rocsm","server-args"],"backgroundTag":"cuda-only-feature-on-non-cuda-platform","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}