vllm-project/vllm · error · ValueError

got {len(layers)} per-layer configs for a model with {merged

Error message

got {len(layers)} per-layer configs for a model with {merged['total_num_hidden_layers']} layers

What it means

After merging, from_layers cross-checks that the number of supplied per-layer configs equals total_num_hidden_layers in the merged config. A mismatch means the extracted layer list and the declared depth disagree — the merged config would describe the wrong shape.

Source

Thrown at vllm/config/model_arch.py:137

            # `bool` is an `int`, so an exact type check is what keeps a varying
            # flag from collapsing to `any`. `is_deepseek_mla` doing that would
            # make `use_mla` true model wide, and `get_num_kv_heads` then returns
            # 1 for every layer, discarding the overrides built here.
            if not all(type(value) in (int, float) for value in values):
                raise ValueError(
                    f"{f.name!r} varies across layers and has no whole-model "
                    f"value: {sorted(set(map(repr, values)))}. Only numeric "
                    f"fields collapse (with `max`, to size buffers for the "
                    f"largest layer); give this one an explicit rule in "
                    f"ModelArchitectureConfig.from_layers."
                )
            merged[f.name] = max(values)
            for override, value in zip(overrides, values):
                if value != merged[f.name]:
                    override[f.name] = value

        if len(layers) != merged["total_num_hidden_layers"]:
            raise ValueError(
                f"got {len(layers)} per-layer configs for a model with "
                f"{merged['total_num_hidden_layers']} layers"
            )
        # A checkpoint can be heterogeneous over attributes vLLM never reads, in
        # which case there is nothing to keep the layers apart for.
        return cls(**merged, per_layer_overrides=overrides if any(overrides) else None)

View on GitHub (pinned to c794754062)

Solutions

1. Fix the layer extraction to emit exactly total_num_hidden_layers configs (account for tied/shared layers and MoE structure).
2. Verify the checkpoint's declared num_hidden_layers matches the actual weights present (re-download or re-convert if shards are missing).
3. If depth legitimately varies, give total_num_hidden_layers an explicit rule in from_layers rather than relying on max collapse.

Example fix

# before
layer_cfgs = [cfg for cfg in all_cfgs if 'self_attn' in cfg]  # dropped hybrid layers
# after
layer_cfgs = all_cfgs[:merged_total_num_hidden_layers]
ModelArchitectureConfig.from_layers(layer_cfgs)

def layer_count_ok(layer_cfgs, expected: int) -> bool:
    return len(layer_cfgs) == expected
# compare against the checkpoint's declared num_hidden_layers before from_layers

def matches_declared_depth(layer_cfgs: list, declared: int) -> bool:
    return len(layer_cfgs) == declared

except ValueError as e:
    if 'per-layer configs for a model with' in str(e):
        re-extract layers with a corrected filter (include tied/shared and MoE layers) and retry the build

When it happens

Trigger: from_layers receives N configs while merged['total_num_hidden_layers'] is M != N — e.g. layer extraction dropped/added layers (regex missing tied or MoE expert layers), or total_num_hidden_layers itself varies and got max-collapsed.

Common situations: Checkpoints with shared/tied layer weights where extraction skips duplicates; safetensors indexes listing partial shards; heterogeneous checkpoints whose total_num_hidden_layers differs per component and max() picks the largest.

Related errors

• {f.name!r} varies across layers and has no whole-model value
• a model must have at least one layer
• mtp_layer_types must have one entry per MTP layer: got {len(
• The Inkling checkpoint does not contain MTP weights
• Unexpected socket type: {socket_type}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/9acb43cd57125f83. Report an issue: GitHub.