{"record":{"id":"0f1c8a499a46c082","repo":"vllm-project/vllm","slug":"f-name-r-varies-across-layers-and-has-no-whole-m","errorCode":null,"errorMessage":"{f.name!r} varies across layers and has no whole-model value: {sorted(set(map(repr, values)))}. Only numeric fields collapse (with `max`, to size buffers for the largest layer); give this one an explicit rule in ModelArchitectureConfig.from_layers.","messagePattern":"(.+?) varies across layers and has no whole-model value: (.+?)\\. Only numeric fields collapse \\(with `max`, to size buffers for the largest layer\\); give this one an explicit rule in ModelArchitectureConfig\\.from_layers\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/model_arch.py","lineNumber":124,"sourceCode":"        \"\"\"\n        if not layers:\n            raise ValueError(\"a model must have at least one layer\")\n\n        merged: dict[str, Any] = {}\n        overrides: list[dict[str, Any]] = [{} for _ in layers]\n        for f in dataclass_fields(cls):\n            if f.name == \"per_layer_overrides\":\n                continue\n            values = [getattr(layer, f.name) for layer in layers]\n            if all(value == values[0] for value in values):\n                merged[f.name] = values[0]\n                continue\n            # `bool` is an `int`, so an exact type check is what keeps a varying\n            # flag from collapsing to `any`. `is_deepseek_mla` doing that would\n            # make `use_mla` true model wide, and `get_num_kv_heads` then returns\n            # 1 for every layer, discarding the overrides built here.\n            if not all(type(value) in (int, float) for value in values):\n                raise ValueError(\n                    f\"{f.name!r} varies across layers and has no whole-model \"\n                    f\"value: {sorted(set(map(repr, values)))}. Only numeric \"\n                    f\"fields collapse (with `max`, to size buffers for the \"\n                    f\"largest layer); give this one an explicit rule in \"\n                    f\"ModelArchitectureConfig.from_layers.\"\n                )\n            merged[f.name] = max(values)\n            for override, value in zip(overrides, values):\n                if value != merged[f.name]:\n                    override[f.name] = value\n\n        if len(layers) != merged[\"total_num_hidden_layers\"]:\n            raise ValueError(\n                f\"got {len(layers)} per-layer configs for a model with \"\n                f\"{merged['total_num_hidden_layers']} layers\"\n            )\n        # A checkpoint can be heterogeneous over attributes vLLM never reads, in\n        # which case there is nothing to keep the layers apart for.","sourceCodeStart":106,"sourceCodeEnd":142,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model_arch.py#L106-L142","documentation":"from_layers only auto-collapses numeric fields with max (to size buffers for the largest layer). If a non-numeric field (str, bool-that-varies via exact type check, nested object) differs across layers, there is no safe whole-model value, so it raises and asks for an explicit merge rule.","triggerScenarios":"A heterogeneous checkpoint where e.g. intermediate_size or a string/enum field (rope type, attention variant) differs between layers, passed through from_layers; the exact-type check deliberately excludes bools from int collapsing.","commonSituations":"Custom hybrid or layerwise-pruned checkpoints where a string field differs per layer; new ModelArchitectureConfig fields added without a from_layers merge rule; converting MoE/hybrid checkpoints with per-layer rope scaling types.","solutions":["Make the varying field uniform across layers in the checkpoint (re-convert so all layers share one value).","Add an explicit merge rule for that field in ModelArchitectureConfig.from_layers (source change) as the error message instructs.","If the variation is spurious (metadata noise), normalize the HF configs before extraction so values agree."],"exampleFix":"// before: per-layer configs disagree on rope_scaling type\nlayers[0].rope_scaling = {\"type\": \"linear\"}\nlayers[1].rope_scaling = {\"type\": \"dynamic\"}\n// after: normalize before merge\nfor l in layers: l.rope_scaling = {\"type\": \"linear\"}\nModelArchitectureConfig.from_layers(layers)","handlingStrategy":"validation","validationCode":"from dataclasses import fields\nimport ModelArchitectureConfig  # your import path\ndef mergeable(layer_cfgs) -> bool:\n    numeric = {int, float}\n    for f in fields(ModelArchitectureConfig):\n        if f.name == 'per_layer_overrides':\n            continue\n        vals = [getattr(l, f.name) for l in layer_cfgs]\n        if any(v != vals[0] for v in vals) and not all(type(v) in numeric for v in vals):\n            return False\n    return True","typeGuard":null,"tryCatchPattern":"except ValueError as e:\n    if 'varies across layers' in str(e):\n        report the field name from the message and normalize that field in the checkpoint configs","preventionTips":["Normalize non-numeric HF config fields to a single value across layers before conversion.","When adding fields to ModelArchitectureConfig, decide their from_layers merge rule at the same time.","Remember bools are deliberately not int-collapsed; make boolean flags uniform across layers."],"tags":["model-arch","heterogeneous-layers","internal-api"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}