{"record":{"id":"0cfa6dd7b9d10bcc","repo":"sgl-project/sglang","slug":"mlx-model-has-no-supported-attention-layers","errorCode":null,"errorMessage":"MLX model has no supported attention layers","messagePattern":"MLX model has no supported attention layers","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/hardware_backend/mlx/kv_cache/layout.py","lineNumber":108,"sourceCode":"    def num_attention_layers(self) -> int:\n        return len(self.attention_layer_indices)\n\n    @property\n    def num_full_attention_layers(self) -> int:\n        return len(self.full_attention_layer_indices)\n\n    @property\n    def has_sliding_window_layers(self) -> bool:\n        return bool(self.swa_attention_layer_indices)\n\n    @property\n    def has_auxiliary_state(self) -> bool:\n        return bool(self.auxiliary_layer_indices)\n\n    @property\n    def first_attention_layer_index(self) -> int:\n        if not self.attention_layer_indices:\n            raise RuntimeError(\"MLX model has no supported attention layers\")\n        return self.attention_layer_indices[0]\n\n    def window_size(self, layer_idx: int) -> int | None:\n        \"\"\"Sliding window of *layer_idx*, or None for full attention.\"\"\"\n        return self.layer_window_sizes.get(layer_idx)\n\n    def attention_pool_index(self, layer_idx: int) -> int:\n        try:\n            return self.attention_pool_index_by_layer[layer_idx]\n        except KeyError as exc:\n            raise KeyError(f\"Layer {layer_idx} is not an attention layer\") from exc\n\n    def full_kv_pool_index(self, layer_idx: int) -> int:\n        try:\n            return self.full_kv_pool_index_by_layer[layer_idx]\n        except KeyError as exc:\n            raise KeyError(f\"Layer {layer_idx} is not a full-attention layer\") from exc\n","sourceCodeStart":90,"sourceCodeEnd":126,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/hardware_backend/mlx/kv_cache/layout.py#L90-L126","documentation":"`MlxModelCacheLayout.first_attention_layer_index` returns the index of the first attention layer to seed things like the KV pool. If the layout has no attention layers at all (all attrs were None during discovery), there is no meaningful answer and it raises RuntimeError.","triggerScenarios":"Accessing first_attention_layer_index on a layout built with all-None attention_attrs — e.g. discovery failed to recognize any attention module (unsupported architecture) or a Mamba-only model was passed.","commonSituations":"Running a pure linear-attention/Mamba model through the MLX KV-cache layout path; a rename in mlx_lm so no attention attrs match; wrapping a model whose attention layers live in a non-standard container.","solutions":["Check `layout.attention_layer_indices` (or has_attention) before accessing the property.","Fix attention discovery for the model so attention_attrs contains real attribute names.","For Mamba-only models, use the auxiliary-state path instead of the KV-cache layout path."],"exampleFix":"# before\nidx = layout.first_attention_layer_index  # RuntimeError\n\n# after\nidx = (\n    layout.first_attention_layer_index\n    if layout.attention_layer_indices\n    else None\n)","handlingStrategy":"type-guard","validationCode":"if not layout.attention_layer_indices:\n    raise ValueError(\"no attention layers discovered; check discovery attrs\")\nidx = layout.first_attention_layer_index","typeGuard":"def has_attention(layout) -> bool:\n    return bool(layout.attention_layer_indices)","tryCatchPattern":null,"preventionTips":["Validate discovery results (at least one attention attr) before building the layout.","Route Mamba-only models to the auxiliary-state path."],"tags":["mlx","kv-cache","layout","model-compatibility"],"backgroundTag":"empty-collection-access","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}