{"record":{"id":"4cf133126fe63864","repo":"sgl-project/sglang","slug":"model-model-cls-does-not-support-custom-attentio","errorCode":null,"errorMessage":"Model {model_cls} does not support custom attention backends (_supports_attention_backend=False). The Transformers backend requires custom attention support.","messagePattern":"Model (.+?) does not support custom attention backends \\(_supports_attention_backend=False\\)\\. The Transformers backend requires custom attention support\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/transformers.py","lineNumber":622,"sourceCode":"        model_cls = _resolve_attention_backend_model_cls(config)\n\n        supports_backend = (\n            getattr(model_cls, \"_supports_attention_backend\", True)\n            if model_cls\n            else True\n        )\n\n        # Initialize on meta device to avoid premature GPU allocation\n        self.text_config._attn_implementation = \"sglang\"\n        if supports_backend:\n            with _init_on_device_without_buffers(torch.device(\"meta\")):\n                self.model: PreTrainedModel = AutoModel.from_config(\n                    self.config,\n                    torch_dtype=torch.get_default_dtype(),\n                    trust_remote_code=True,\n                )\n        else:\n            raise ValueError(\n                f\"Model {model_cls} does not support custom attention backends \"\n                \"(_supports_attention_backend=False). The Transformers backend \"\n                \"requires custom attention support.\"\n            )\n\n        self.vocab_size = getattr(\n            self.text_config,\n            \"vocab_size\",\n            self.model.get_input_embeddings().num_embeddings,\n        )\n        self.unpadded_vocab_size = self.vocab_size\n\n        # Embedding scale (e.g. Whisper)\n        input_embeddings = self.model.get_input_embeddings()\n        self.embed_scale = getattr(input_embeddings, \"embed_scale\", None)\n\n        self.start_layer = 0\n        self.end_layer = getattr(self.text_config, \"num_hidden_layers\", 0)","sourceCodeStart":604,"sourceCodeEnd":640,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/transformers.py#L604-L640","documentation":"The Transformers (AutoModel) backend requires the underlying HF model class to advertise custom attention backend support (_supports_attention_backend). If the flag is absent/False, instantiation fails because sglang cannot hook its attention.","triggerScenarios":"Loading any HF architecture whose model class has not set _supports_attention_backend=True (most pre-transformers>=4.56 classes).","commonSituations":"Trying to serve a brand-new or older HF model through the Transformers fallback backend; transformers version too old so the flag doesn't exist.","solutions":["Use the model's native sglang implementation if one exists","Upgrade transformers to a version where the model sets _supports_attention_backend=True","Avoid the Transformers backend for this model (select architecture-specific path)"],"exampleFix":"# before: unsupported model via Transformers backend\nModelForCausalLM (_supports_attention_backend unset)\n# after (model side / subclass)\nclass MyModel(...):\n    _supports_attention_backend = True","handlingStrategy":"validation","validationCode":"ok = getattr(HFModelCls, '_supports_attention_backend', False)\nif not ok: raise SystemExit('use native sglang impl for this model')","typeGuard":"def backend_compatible(cls) -> bool:\n    return bool(getattr(cls, '_supports_attention_backend', False))","tryCatchPattern":null,"preventionTips":["Prefer native sglang model implementations","Check the flag before choosing the Transformers backend"],"tags":["transformers-backend","attention-backend","compatibility"],"backgroundTag":"unsupported-model-backend","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}