sgl-project/sglang · error · ValueError

CuteDSL MLA backend only supports kv-cache-dtype of fp8_e4m3

Error message

CuteDSL MLA backend only supports kv-cache-dtype of fp8_e4m3, bf16, or auto.

What it means

The cutedsl_mla prefill backend only supports KV cache dtypes fp8_e4m3, bf16/bfloat16, or auto; its CUTLASS DSL kernels are not instantiated for other element types (e.g. fp16, fp8_e5m2, fp4).

Source

Thrown at python/sglang/srt/arg_groups/overrides.py:2550

        view.attention_backend == "cutedsl_mla"
        or view.decode_attention_backend == "cutedsl_mla"
        or view.prefill_attention_backend == "cutedsl_mla"
    ):
        return {}
    assert (
        view.prefill_attention_backend != "cutedsl_mla"
    ), "CuteDSL MLA only supports decoding for now"
    if not is_sm100_supported():
        raise ValueError(
            "CuteDSL MLA backend is only supported on Blackwell GPUs (SM100). Please use a different backend."
        )
    if view.kv_cache_dtype not in [
        "fp8_e4m3",
        "bf16",
        "bfloat16",
        "auto",
    ]:
        raise ValueError(
            "CuteDSL MLA backend only supports kv-cache-dtype of fp8_e4m3, bf16, or auto."
        )
    if view.prefill_attention_backend is None:
        return {"prefill_attention_backend": "trtllm_mla"}
    return {}


@register_post_process
def _attention_backend_fa3_fp8_fallback(view: Any) -> dict:
    if view.attention_backend == "fa3" and view.kv_cache_dtype == "fp8_e5m2":
        logger.warning(
            "FlashAttention3 only supports fp8_e4m3 if using FP8; "
            "Setting attention backend to triton."
        )
        return {"attention_backend": "triton"}
    return {}

View on GitHub (pinned to 0132848349)

Solutions

1. Set --kv-cache-dtype fp8_e4m3 or bf16 (or omit it to use auto)
2. Use a different prefill backend if the dtype is a hard requirement

Example fix

# before
--prefill-attention-backend cutedsl_mla --kv-cache-dtype fp8_e5m2
# after
--prefill-attention-backend cutedsl_mla --kv-cache-dtype fp8_e4m3

allowed = {"fp8_e4m3","bf16","bfloat16","auto"}
if args.prefill_attention_backend == "cutedsl_mla" and args.kv_cache_dtype not in allowed:
    args.kv_cache_dtype = "fp8_e4m3"

When it happens

Trigger: --prefill-attention-backend cutedsl_mla together with --kv-cache-dtype outside {fp8_e4m3, bf16, bfloat16, auto}.

Common situations: Porting an fp16 or e5m2 cache config from another backend onto cutedsl_mla prefill.

Related errors

• TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8
• tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, got
• HiSparse requires one of {HISPARSE_KV_CACHE_DTYPES} KV cache
• CuteDSL MLA backend is only supported on Blackwell GPUs (SM1
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b4a2d7aa4c006a4e. Report an issue: GitHub.