{"record":{"id":"3d8a6a30000bbd7e","repo":"sgl-project/sglang","slug":"layer-count-and-attention-attribute-count-differ","errorCode":null,"errorMessage":"Layer count and attention attribute count differ: {len(layers)} != {len(attention_attrs)}","messagePattern":"Layer count and attention attribute count differ: (.+?) != (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/hardware_backend/mlx/kv_cache/layout.py","lineNumber":60,"sourceCode":"            if self.layer_window_sizes.get(idx) is not None\n        )\n        object.__setattr__(self, \"full_attention_layer_indices\", full_indices)\n        object.__setattr__(self, \"swa_attention_layer_indices\", swa_indices)\n        object.__setattr__(\n            self,\n            \"full_kv_pool_index_by_layer\",\n            {layer_idx: pool_idx for pool_idx, layer_idx in enumerate(full_indices)},\n        )\n\n    @classmethod\n    def from_attention_discovery(\n        cls,\n        layers: Sequence[Any],\n        attention_attrs: Sequence[str | None],\n        layer_window_sizes: dict[int, int | None] | None = None,\n    ) -> MlxModelCacheLayout:\n        if len(layers) != len(attention_attrs):\n            raise ValueError(\n                \"Layer count and attention attribute count differ: \"\n                f\"{len(layers)} != {len(attention_attrs)}\"\n            )\n\n        attention_layer_indices = tuple(\n            idx for idx, attr in enumerate(attention_attrs) if attr is not None\n        )\n        auxiliary_layer_indices = tuple(\n            idx for idx, attr in enumerate(attention_attrs) if attr is None\n        )\n        attention_pool_index_by_layer = {\n            layer_idx: pool_idx\n            for pool_idx, layer_idx in enumerate(attention_layer_indices)\n        }\n\n        return cls(\n            layers=tuple(layers),\n            attention_attrs=tuple(attention_attrs),","sourceCodeStart":42,"sourceCodeEnd":78,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/hardware_backend/mlx/kv_cache/layout.py#L42-L78","documentation":"`MlxModelCacheLayout.from_attention_discovery` builds a layout by zipping model layers with a parallel sequence of attention attribute names (None for non-attention layers). The two sequences must be exactly the same length; a mismatch means the discovery routine returned fewer/more attrs than there are layers, and the layout would misalign layer indices.","triggerScenarios":"Calling from_attention_discovery with a hand-built or discovery-bug-produced attention_attrs list whose length differs from len(layers) — e.g. iterating model.layers but collecting attrs with a filtered comprehension, or a model with a non-standard layer container.","commonSituations":"Adding support for a new model architecture where layers are nested or named differently; refactoring the discovery code; off-by-one when appending a trailing non-attention layer.","solutions":["Fix the discovery code so attention_attrs has one entry per layer (use None for non-attention layers).","Log/assert len equality in your discovery helper before calling from_attention_discovery.","For new architectures, map attr names exactly onto the model's layer list order."],"exampleFix":"# before\nattrs = [name for name in discover(model) if name]  # filtered -> shorter\nlayout = MlxModelCacheLayout.from_attention_discovery(model.layers, attrs)\n\n# after\nattrs = [discover_attr(layer) for layer in model.layers]  # None where absent\nlayout = MlxModelCacheLayout.from_attention_discovery(model.layers, attrs)","handlingStrategy":"validation","validationCode":"assert len(layers) == len(attention_attrs), (\n    f\"{len(layers)} layers vs {len(attention_attrs)} attrs\"\n)\nlayout = MlxModelCacheLayout.from_attention_discovery(layers, attention_attrs)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always build attention_attrs with a per-layer comprehension, never filtered.","Add a model-load smoke test that constructs the layout for each supported arch."],"tags":["mlx","kv-cache","layout","validation"],"backgroundTag":"length-mismatch-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}