huggingface/transformers · error · ValueError

The following attributes are missing: {sorted(missing_requir

Error message

The following attributes are missing: {sorted(missing_required_global_attributes)}
Please define them globally, or provide them for every layer in `per_layer_config`

What it means

For every attribute that appears in some (but not necessarily all) per_layer_config entries, the attribute must be resolvable everywhere: either defined globally on the config, or present in every layer's override (and if per-layer coverage is partial — fewer entries than num_hidden_layers — only the global route works). Attributes failing this are collected and reported as missing, because a middle layer with no override and no global value would have an undefined attribute.

Source

Thrown at src/transformers/integrations/heterogeneity/configuration_utils.py:133

    config: PreTrainedConfig, per_layer_overrides: dict[int, dict[str, Any]]
) -> _HeterogeneitySpec:
    explicit_per_layer_attributes = _get_per_layer_attributes(per_layer_overrides)

    # Ensure all required global attributes are defined
    missing_required_global_attributes = set()
    for attr in explicit_per_layer_attributes:
        if len(per_layer_overrides) != config.num_hidden_layers:
            if not config._hasattr_without_heterogeneous_validation(attr):
                missing_required_global_attributes.add(attr)
        else:
            for layer_overrides in per_layer_overrides.values():
                if attr not in layer_overrides:
                    if not config._hasattr_without_heterogeneous_validation(attr):
                        missing_required_global_attributes.add(attr)
                    break

    if missing_required_global_attributes:
        raise ValueError(
            f"The following attributes are missing: {sorted(missing_required_global_attributes)}\nPlease define them globally, or provide them for every layer in `per_layer_config`"
        )

    # Remove per-layer overrides that match the global value
    for attr in explicit_per_layer_attributes:
        if not config._hasattr_without_heterogeneous_validation(attr):
            continue

        global_value = config._getattr_without_heterogeneous_validation(attr)
        for layer_overrides in per_layer_overrides.values():
            if attr in layer_overrides and layer_overrides[attr] == global_value:
                del layer_overrides[attr]

    # Delete all empty layer configs
    for layer_idx, layer_overrides in list(per_layer_overrides.items()):
        if not layer_overrides:
            del per_layer_overrides[layer_idx]

View on GitHub (pinned to a597f97485)

Solutions

1. Define the listed attributes globally on the config (e.g. config.sliding_window = 512) as a safe default
2. Or add the attribute to every layer's per_layer_config entry so no layer is left without a value
3. If coverage is partial, prefer the global-default route — per-layer-only attributes require full coverage

Example fix

# before: attribute only on some layers, no global default
config.per_layer_config = {0: {"rope_local_base_freq": 10000.0}}
# ValueError: The following attributes are missing: ['rope_local_base_freq']

# after: define a global default
config.rope_local_base_freq = 10000.0
config.per_layer_config = {0: {"rope_local_base_freq": 10000.0}}

def validate_per_layer_attrs(config, per_layer):
    attrs = {a for ov in per_layer.values() for a in ov}
    missing = [
        a for a in sorted(attrs)
        if not hasattr(config, a)
        and (len(per_layer) != config.num_hidden_layers or any(a not in ov for ov in per_layer.values()))
    ]
    assert not missing, f"define these globally or per-layer everywhere: {missing}"

validate_per_layer_attrs(config, per_layer_config)

When it happens

Trigger: per_layer_config covering a subset of layers (len(per_layer_overrides) != num_hidden_layers) that introduces a new attribute not present on the global config; or full coverage where one layer omits an attribute that others set and the global config lacks it.

Common situations: Adding per-layer knobs like rope scaling or head counts for a few layers without defining a global default in the PretrainedConfig; partial recipes copied from heterogeneous model implementations (mix'n'match layers) missing the base attribute.

Related errors

• `skip` must be an iterable of strings.
• `skip` must contain only strings.
• `per_layer_config` keys must be integer layer indices in the
• Layer type '{layer_idx}' not found in config.layer_types: {l
• Layer type '{layer_idx}' is not homogeneous across layers (l

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/9d5750d2d63942b9. Report an issue: GitHub.