sgl-project/sglang · error · ValueError

Model {model_cls} does not support custom attention backends

Error message

Model {model_cls} does not support custom attention backends (_supports_attention_backend=False). The Transformers backend requires custom attention support.

What it means

The Transformers (AutoModel) backend requires the underlying HF model class to advertise custom attention backend support (_supports_attention_backend). If the flag is absent/False, instantiation fails because sglang cannot hook its attention.

Source

Thrown at python/sglang/srt/models/transformers.py:622

        model_cls = _resolve_attention_backend_model_cls(config)

        supports_backend = (
            getattr(model_cls, "_supports_attention_backend", True)
            if model_cls
            else True
        )

        # Initialize on meta device to avoid premature GPU allocation
        self.text_config._attn_implementation = "sglang"
        if supports_backend:
            with _init_on_device_without_buffers(torch.device("meta")):
                self.model: PreTrainedModel = AutoModel.from_config(
                    self.config,
                    torch_dtype=torch.get_default_dtype(),
                    trust_remote_code=True,
                )
        else:
            raise ValueError(
                f"Model {model_cls} does not support custom attention backends "
                "(_supports_attention_backend=False). The Transformers backend "
                "requires custom attention support."
            )

        self.vocab_size = getattr(
            self.text_config,
            "vocab_size",
            self.model.get_input_embeddings().num_embeddings,
        )
        self.unpadded_vocab_size = self.vocab_size

        # Embedding scale (e.g. Whisper)
        input_embeddings = self.model.get_input_embeddings()
        self.embed_scale = getattr(input_embeddings, "embed_scale", None)

        self.start_layer = 0
        self.end_layer = getattr(self.text_config, "num_hidden_layers", 0)

View on GitHub (pinned to 0132848349)

Solutions

1. Use the model's native sglang implementation if one exists
2. Upgrade transformers to a version where the model sets _supports_attention_backend=True
3. Avoid the Transformers backend for this model (select architecture-specific path)

Example fix

# before: unsupported model via Transformers backend
ModelForCausalLM (_supports_attention_backend unset)
# after (model side / subclass)
class MyModel(...):
    _supports_attention_backend = True

ok = getattr(HFModelCls, '_supports_attention_backend', False)
if not ok: raise SystemExit('use native sglang impl for this model')

def backend_compatible(cls) -> bool:
    return bool(getattr(cls, '_supports_attention_backend', False))

When it happens

Trigger: Loading any HF architecture whose model class has not set _supports_attention_backend=True (most pre-transformers>=4.56 classes).

Common situations: Trying to serve a brand-new or older HF model through the Transformers fallback backend; transformers version too old so the flag doesn't exist.

Related errors

• flashinfer_sparse_mla supports only GLM DSA with FP8 KV cach
• GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports onl
• MiniMax-H3 ring parallelism requires the FlashAttention back
• AITer backend does not have a metadata builder.
• AITer backend requires num_heads ({num_heads}) to be a multi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4cf133126fe63864. Report an issue: GitHub.