{"record":{"id":"50f82a4b5a705b9a","repo":"sgl-project/sglang","slug":"tokenspeed-mla-backend-requires-kv-cache-dtype-fp8","errorCode":null,"errorMessage":"tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, got {}.","messagePattern":"tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/overrides.py","lineNumber":2507,"sourceCode":"    ):\n        if not is_blackwell_supported():\n            raise ValueError(\n                \"TRTLLM MLA backend is only supported on Blackwell GPUs (SM100/SM12x). Please use a different backend.\"\n            )\n        if view.kv_cache_dtype not in [\"fp8_e4m3\", \"fp4_e2m1\", \"bf16\", \"auto\"]:\n            raise ValueError(\n                \"TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8_e4m3, fp4_e2m1, bf16, or auto.\"\n            )\n    if (\n        view.attention_backend == \"tokenspeed_mla\"\n        or view.decode_attention_backend == \"tokenspeed_mla\"\n    ):\n        if not is_blackwell_supported():\n            raise ValueError(\n                \"tokenspeed_mla backend is only supported on Blackwell GPUs (SM100/SM12x).\"\n            )\n        if view.kv_cache_dtype not in [\"fp8_e4m3\"]:\n            raise ValueError(\n                \"tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, \"\n                f\"got {view.kv_cache_dtype}.\"\n            )\n    return {}\n\n\n@register_post_process\ndef _hisparse_validation(view: Any) -> dict:\n    \"\"\"Read-only validation pass: --enable-hisparse constraints (model class,\n    radix cache, kv dtype, DSA backends) read the resolved values through the\n    view.\"\"\"\n    from sglang.srt.arg_groups.hisparse_hook import validate_hisparse\n\n    validate_hisparse(view)\n    return {}\n\n\n@register_post_process","sourceCodeStart":2489,"sourceCodeEnd":2525,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/overrides.py#L2489-L2525","documentation":"The tokenspeed_mla backend in SGLang requires the KV cache to be stored in fp8_e4m3; its fused kernels do not support bf16/fp16/fp4 or auto resolution to those types. Validation fails at startup when any other dtype is configured.","triggerScenarios":"--attention-backend tokenspeed_mla (or --decode-attention-backend tokenspeed_mla) on Blackwell with --kv-cache-dtype set to bf16, fp16, auto-resolved-bf16, or fp4.","commonSituations":"Reusing a bf16 DeepSeek MLA config when experimenting with tokenspeed_mla; leaving kv-cache-dtype to 'auto' when the model default resolves to bf16.","solutions":["Set --kv-cache-dtype fp8_e4m3 explicitly","If fp8 KV cache is unacceptable (accuracy concerns), use a different MLA backend such as flashmla or trtllm_mla with bf16"],"exampleFix":"# before\n--attention-backend tokenspeed_mla --kv-cache-dtype bf16\n# after\n--attention-backend tokenspeed_mla --kv-cache-dtype fp8_e4m3","handlingStrategy":"validation","validationCode":"if \"tokenspeed_mla\" in (args.attention_backend, args.decode_attention_backend):\n    args.kv_cache_dtype = \"fp8_e4m3\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat tokenspeed_mla as implying fp8_e4m3 in config templates"],"tags":["sglang","tokenspeed","mla","kv-cache-dtype","config-validation"],"backgroundTag":"unsupported-argument-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}