sgl-project/sglang · error · ValueError

Layer count and attention attribute count differ: {len(layer

Error message

Layer count and attention attribute count differ: {len(layers)} != {len(attention_attrs)}

What it means

`MlxModelCacheLayout.from_attention_discovery` builds a layout by zipping model layers with a parallel sequence of attention attribute names (None for non-attention layers). The two sequences must be exactly the same length; a mismatch means the discovery routine returned fewer/more attrs than there are layers, and the layout would misalign layer indices.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/kv_cache/layout.py:60

            if self.layer_window_sizes.get(idx) is not None
        )
        object.__setattr__(self, "full_attention_layer_indices", full_indices)
        object.__setattr__(self, "swa_attention_layer_indices", swa_indices)
        object.__setattr__(
            self,
            "full_kv_pool_index_by_layer",
            {layer_idx: pool_idx for pool_idx, layer_idx in enumerate(full_indices)},
        )

    @classmethod
    def from_attention_discovery(
        cls,
        layers: Sequence[Any],
        attention_attrs: Sequence[str | None],
        layer_window_sizes: dict[int, int | None] | None = None,
    ) -> MlxModelCacheLayout:
        if len(layers) != len(attention_attrs):
            raise ValueError(
                "Layer count and attention attribute count differ: "
                f"{len(layers)} != {len(attention_attrs)}"
            )

        attention_layer_indices = tuple(
            idx for idx, attr in enumerate(attention_attrs) if attr is not None
        )
        auxiliary_layer_indices = tuple(
            idx for idx, attr in enumerate(attention_attrs) if attr is None
        )
        attention_pool_index_by_layer = {
            layer_idx: pool_idx
            for pool_idx, layer_idx in enumerate(attention_layer_indices)
        }

        return cls(
            layers=tuple(layers),
            attention_attrs=tuple(attention_attrs),

View on GitHub (pinned to 0132848349)

Solutions

1. Fix the discovery code so attention_attrs has one entry per layer (use None for non-attention layers).
2. Log/assert len equality in your discovery helper before calling from_attention_discovery.
3. For new architectures, map attr names exactly onto the model's layer list order.

Example fix

# before
attrs = [name for name in discover(model) if name]  # filtered -> shorter
layout = MlxModelCacheLayout.from_attention_discovery(model.layers, attrs)

# after
attrs = [discover_attr(layer) for layer in model.layers]  # None where absent
layout = MlxModelCacheLayout.from_attention_discovery(model.layers, attrs)

assert len(layers) == len(attention_attrs), (
    f"{len(layers)} layers vs {len(attention_attrs)} attrs"
)
layout = MlxModelCacheLayout.from_attention_discovery(layers, attention_attrs)

When it happens

Trigger: Calling from_attention_discovery with a hand-built or discovery-bug-produced attention_attrs list whose length differs from len(layers) — e.g. iterating model.layers but collecting attrs with a filtered comprehension, or a model with a non-standard layer container.

Common situations: Adding support for a new model architecture where layers are nested or named differently; refactoring the discovery code; off-by-one when appending a trailing non-attention layer.

Related errors

• MLX model has no supported attention layers
• v_cache must be provided
• kv-canary: {name} must be on {reference_name}'s device {refe
• kv-canary: launch_canary_plan_kernels_torch_reference verify
• kv-canary: launch_canary_plan_kernels_torch_reference requir

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3d8a6a30000bbd7e. Report an issue: GitHub.