{"record":{"id":"f18c7949652c1042","repo":"sgl-project/sglang","slug":"tensorrt-llm-mla-backend-only-supports-kv-cache-dt","errorCode":null,"errorMessage":"TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8_e4m3, fp4_e2m1, bf16, or auto.","messagePattern":"TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8_e4m3, fp4_e2m1, bf16, or auto\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/overrides.py","lineNumber":2495,"sourceCode":"    return {}\n\n\n@register_post_process\ndef _mla_kv_cache_dtype_checks(view: Any) -> dict:\n    \"\"\"Read-only validation pass in the attention-backend compatibility\n    handler: the TRT-LLM and tokenspeed MLA backends constrain the resolved\n    kv-cache dtype (declarations never reach the field, so the checks read\n    the view).\"\"\"\n    if (\n        view.attention_backend == \"trtllm_mla\"\n        or view.decode_attention_backend == \"trtllm_mla\"\n    ):\n        if not is_blackwell_supported():\n            raise ValueError(\n                \"TRTLLM MLA backend is only supported on Blackwell GPUs (SM100/SM12x). Please use a different backend.\"\n            )\n        if view.kv_cache_dtype not in [\"fp8_e4m3\", \"fp4_e2m1\", \"bf16\", \"auto\"]:\n            raise ValueError(\n                \"TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8_e4m3, fp4_e2m1, bf16, or auto.\"\n            )\n    if (\n        view.attention_backend == \"tokenspeed_mla\"\n        or view.decode_attention_backend == \"tokenspeed_mla\"\n    ):\n        if not is_blackwell_supported():\n            raise ValueError(\n                \"tokenspeed_mla backend is only supported on Blackwell GPUs (SM100/SM12x).\"\n            )\n        if view.kv_cache_dtype not in [\"fp8_e4m3\"]:\n            raise ValueError(\n                \"tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, \"\n                f\"got {view.kv_cache_dtype}.\"\n            )\n    return {}\n\n","sourceCodeStart":2477,"sourceCodeEnd":2513,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/overrides.py#L2477-L2513","documentation":"Raised by SGLang's server-args validation when the TensorRT-LLM MLA attention backend is selected but --kv-cache-dtype is not one of fp8_e4m3, fp4_e2m1, bf16, or auto. The TRTLLM MLA kernels are only compiled for those cache element types, so any other dtype (e.g. fp16, fp8_e5m2) is rejected at startup.","triggerScenarios":"Setting --attention-backend trtllm_mla (or --decode-attention-backend trtllm_mla) on a Blackwell GPU together with --kv-cache-dtype set to anything besides fp8_e4m3/fp4_e2m1/bf16/auto.","commonSituations":"Copying a DeepSeek serving config tuned for a different backend (fa3, flashmla) that used fp16 or e5m2 KV cache while switching to trtllm_mla on B200/GB200.","solutions":["Set --kv-cache-dtype fp8_e4m3 (or bf16, fp4_e2m1, or auto) when using trtllm_mla","Drop the explicit --kv-cache-dtype and let 'auto' resolve it","If you need another dtype, switch to a different attention backend (e.g. flashmla or fa3)"],"exampleFix":"# before\n--attention-backend trtllm_mla --kv-cache-dtype fp8_e5m2\n# after\n--attention-backend trtllm_mla --kv-cache-dtype fp8_e4m3","handlingStrategy":"validation","validationCode":"allowed = {\"fp8_e4m3\",\"fp4_e2m1\",\"bf16\",\"auto\"}\nif args.attention_backend == \"trtllm_mla\" and args.kv_cache_dtype not in allowed:\n    args.kv_cache_dtype = \"fp8_e4m3\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate backend+dtype pairs in your launch wrapper before spawning sglang","Keep per-backend tuned config files instead of one shared config"],"tags":["sglang","trtllm","mla","kv-cache-dtype","config-validation"],"backgroundTag":"unsupported-argument-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}