vllm-project/vllm · error · ValueError

{f.name!r} varies across layers and has no whole-model value

Error message

{f.name!r} varies across layers and has no whole-model value: {sorted(set(map(repr, values)))}. Only numeric fields collapse (with `max`, to size buffers for the largest layer); give this one an explicit rule in ModelArchitectureConfig.from_layers.

What it means

from_layers only auto-collapses numeric fields with max (to size buffers for the largest layer). If a non-numeric field (str, bool-that-varies via exact type check, nested object) differs across layers, there is no safe whole-model value, so it raises and asks for an explicit merge rule.

Source

Thrown at vllm/config/model_arch.py:124

        """
        if not layers:
            raise ValueError("a model must have at least one layer")

        merged: dict[str, Any] = {}
        overrides: list[dict[str, Any]] = [{} for _ in layers]
        for f in dataclass_fields(cls):
            if f.name == "per_layer_overrides":
                continue
            values = [getattr(layer, f.name) for layer in layers]
            if all(value == values[0] for value in values):
                merged[f.name] = values[0]
                continue
            # `bool` is an `int`, so an exact type check is what keeps a varying
            # flag from collapsing to `any`. `is_deepseek_mla` doing that would
            # make `use_mla` true model wide, and `get_num_kv_heads` then returns
            # 1 for every layer, discarding the overrides built here.
            if not all(type(value) in (int, float) for value in values):
                raise ValueError(
                    f"{f.name!r} varies across layers and has no whole-model "
                    f"value: {sorted(set(map(repr, values)))}. Only numeric "
                    f"fields collapse (with `max`, to size buffers for the "
                    f"largest layer); give this one an explicit rule in "
                    f"ModelArchitectureConfig.from_layers."
                )
            merged[f.name] = max(values)
            for override, value in zip(overrides, values):
                if value != merged[f.name]:
                    override[f.name] = value

        if len(layers) != merged["total_num_hidden_layers"]:
            raise ValueError(
                f"got {len(layers)} per-layer configs for a model with "
                f"{merged['total_num_hidden_layers']} layers"
            )
        # A checkpoint can be heterogeneous over attributes vLLM never reads, in
        # which case there is nothing to keep the layers apart for.

View on GitHub (pinned to c794754062)

Solutions

Make the varying field uniform across layers in the checkpoint (re-convert so all layers share one value).
Add an explicit merge rule for that field in ModelArchitectureConfig.from_layers (source change) as the error message instructs.
If the variation is spurious (metadata noise), normalize the HF configs before extraction so values agree.

Example fix

// before: per-layer configs disagree on rope_scaling type
layers[0].rope_scaling = {"type": "linear"}
layers[1].rope_scaling = {"type": "dynamic"}
// after: normalize before merge
for l in layers: l.rope_scaling = {"type": "linear"}
ModelArchitectureConfig.from_layers(layers)

Defensive patterns

Strategy: validation

Validate before calling

from dataclasses import fields
import ModelArchitectureConfig  # your import path
def mergeable(layer_cfgs) -> bool:
    numeric = {int, float}
    for f in fields(ModelArchitectureConfig):
        if f.name == 'per_layer_overrides':
            continue
        vals = [getattr(l, f.name) for l in layer_cfgs]
        if any(v != vals[0] for v in vals) and not all(type(v) in numeric for v in vals):
            return False
    return True

Try / catch

except ValueError as e:
    if 'varies across layers' in str(e):
        report the field name from the message and normalize that field in the checkpoint configs

Prevention

Normalize non-numeric HF config fields to a single value across layers before conversion.
When adding fields to ModelArchitectureConfig, decide their from_layers merge rule at the same time.
Remember bools are deliberately not int-collapsed; make boolean flags uniform across layers.

When it happens

Trigger: A heterogeneous checkpoint where e.g. intermediate_size or a string/enum field (rope type, attention variant) differs between layers, passed through from_layers; the exact-type check deliberately excludes bools from int collapsing.

Common situations: Custom hybrid or layerwise-pruned checkpoints where a string field differs per layer; new ModelArchitectureConfig fields added without a from_layers merge rule; converting MoE/hybrid checkpoints with per-layer rope scaling types.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/0f1c8a499a46c082. Report an issue: GitHub.