sgl-project/sglang · error · ValueError

Unsupported Kimi-K3 vision attention backend: {attention_bac

Error message

Unsupported Kimi-K3 vision attention backend: {attention_backend}

What it means

MoonViT3dEncoder.__init__ (kimi_k3_vl.py:555) validates the configured multimodal attention backend: only "auto" or one of the entries in QKV_BACKEND_IMPL (e.g. flashinfer, triton, flashinfer_cudnn, fa3) may be used for the Kimi-K3 vision tower. Any other SGLANG_MM_ATTENTION_BACKEND value raises immediately at model build time, before any GPU work.

Source

Thrown at python/sglang/srt/models/kimi_k3_vl.py:555

@dataclass(frozen=True)
class KimiK3VisionForwardMetadata:
    grid_thw_list: Tuple[GridTHW, ...]
    segment_bounds: SegmentBounds
    rope_freqs_cis: torch.Tensor
    attention: VisionAttentionMetadata
    use_fused_rope: bool
    selected_attention_backend: str
    position_embeddings: Optional[torch.Tensor] = None


class MoonViT3dEncoder(nn.Module):
    def __init__(self, hidden_dim: int, num_layers: int, block_cfg: dict) -> None:
        super().__init__()
        qkv_hidden_size = block_cfg.get("qkv_hidden_size") or block_cfg["hidden_dim"]
        attention_backend = _get_mm_attention_backend()
        if attention_backend != "auto" and attention_backend not in QKV_BACKEND_IMPL:
            raise ValueError(
                f"Unsupported Kimi-K3 vision attention backend: {attention_backend}"
            )
        attention_workspace = None
        if attention_backend == "flashinfer_cudnn" and torch.cuda.is_available():
            attention_workspace = torch.empty(
                FLASHINFER_WORKSPACE_SIZE_BYTES,
                dtype=torch.uint8,
                device=torch.device("cuda", torch.cuda.current_device()),
            )
        self.attention_backend = attention_backend
        if attention_backend == "auto":
            print_info_once(
                "Kimi-K3 vision attention uses shape-aware auto selection on "
                "B300/GB300 (Triton for small workloads, FA4 otherwise)."
            )
        self.attention_width = qkv_hidden_size
        self.rope_2d = Rope2DPosEmbRepeated(
            qkv_hidden_size // block_cfg["num_heads"], 512, 512

View on GitHub (pinned to 0132848349)

Solutions

1. Unset the backend (use auto) and let the model pick a supported implementation
2. Set SGLANG_MM_ATTENTION_BACKEND to a value present in QKV_BACKEND_IMPL in this repo version (check the import at the top of kimi_k3_vl.py)
3. Upgrade/downgrade SGLang so the backend name you use exists in this model's allow-list

Example fix

# before
export SGLANG_MM_ATTENTION_BACKEND=torch_native
# after
unset SGLANG_MM_ATTENTION_BACKEND  # or =flashinfer

be = os.getenv("SGLANG_MM_ATTENTION_BACKEND")
assert be is None or be == "auto" or be in QKV_BACKEND_IMPL, be

When it happens

Trigger: Setting a vision attention backend string that has no Kimi-K3 implementation, e.g. SGLANG_MM_ATTENTION_BACKEND=torch_native or an older backend name, then loading the model.

Common situations: Copying server flags/env vars from other models whose backend names differ; upgrading SGLang where backend names were renamed; running on hardware where only a subset of backends is compiled.

Related errors

• Kimi-K3 DCP + DSPARK currently requires SGLANG_RAGGED_VERIFY
• Decode attention backend for Kimi-K3 DCP must be 'cutedsl_ml
• TRTLLM FMHAv2 prefill on SM120 does not support fp8_e4m3 KV
• mega MoE: num_tokens={num_tokens} exceeds SGLANG_OPT_DEEPGEM
• flashinfer_sparse_mla supports only GLM DSA with FP8 KV cach

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a9d7c1e1a87e0aaa. Report an issue: GitHub.