huggingface/transformers · error · AmbiguousGlobalPerLayerAttributeError

'{key}' is a per-layer attribute and may vary across layers.

Error message

'{key}' is a per-layer attribute and may vary across layers. Access it via the individual layer configs instead (e.g. config.per_layer_config[i].{key}). To read the global config value from config.{key} anyway, set `allow_global_per_layer_attribute_access` to `True` on the config. Warning: only do this if the caller can safely handle heterogeneous configs; code that assumes a homogeneous model may use the global value incorrectly.

What it means

On a heterogeneous config, attributes listed in `_heterogeneity_spec.per_layer_attributes` (e.g. `num_key_value_heads`) may differ per layer, so reading them from the global config is ambiguous. The `__getattribute__` hook in the heterogeneity mixin raises `AmbiguousGlobalPerLayerAttributeError` unless the config flag `allow_global_per_layer_attribute_access` is explicitly set to True, in which case it only warns. This forces callers to either read a concrete layer config or consciously opt into the (possibly wrong) global value.

Source

Thrown at src/transformers/integrations/heterogeneity/configuration_utils.py:298

    return explicit_per_layer_overrides


class HeterogeneousConfigMixin:
    """Mixin for heterogeneous per-layer config behavior.

    This mixin owns heterogeneity-specific state and rules. ``PreTrainedConfig`` assigns the ``per_layer_config``
    property in the post-init phase and calls hook methods where heterogeneity needs to participate in the config lifecycle: attribute
    access, key iteration, and serialization.
    """

    def __getattribute__(self, key: str) -> Any:
        # In heterogeneous configs, per-layer attributes are ambiguous on the global config.
        # Callers must read them from a concrete layer unless they explicitly opt into the global value.
        heterogeneity_spec = super().__getattribute__("__dict__").get("_heterogeneity_spec")
        if heterogeneity_spec is not None:
            if key in heterogeneity_spec.per_layer_attributes:
                if not super().__getattribute__("allow_global_per_layer_attribute_access"):
                    raise AmbiguousGlobalPerLayerAttributeError(
                        f"'{key}' is a per-layer attribute and may vary across layers. Access it via the individual layer "
                        f"configs instead (e.g. config.per_layer_config[i].{key}). To read the global config value from "
                        f"config.{key} anyway, set `allow_global_per_layer_attribute_access` to `True` on the config. "
                        f"Warning: only do this if the caller can safely handle heterogeneous configs; code that assumes "
                        f"a homogeneous model may use the global value incorrectly."
                    )

                logger.warning_once(
                    f"Reading global config value for per-layer attribute `{key}` on a heterogeneous config. "
                    "Only do this if the caller can safely handle heterogeneous configs; code that assumes a homogeneous "
                    "model may use the global value incorrectly."
                )

        return super().__getattribute__(key)

    @property
    def is_heterogeneous(self) -> bool:
        return hasattr(self, "_heterogeneity_spec")

View on GitHub (pinned to a597f97485)

Solutions

1. Read the value from a concrete layer: `config.per_layer_config[i].num_key_value_heads`.
2. If your code genuinely handles heterogeneous models, opt in globally on that config instance: `config.allow_global_per_layer_attribute_access = True` (expect a `logger.warning_once`).
3. Update generic introspection code to check `getattr(config, "_heterogeneity_spec", None) is not None` before touching potentially per-layer attributes.

Example fix

# before
kv_heads = config.num_key_value_heads  # AmbiguousGlobalPerLayerAttributeError

# after
kv_heads = config.per_layer_config[0].num_key_value_heads

# or, only if the caller handles heterogeneity safely:
config.allow_global_per_layer_attribute_access = True
kv_heads = config.num_key_value_heads

spec = getattr(config, "_heterogeneity_spec", None)
if spec is not None and key in spec.per_layer_attributes:
    value = config.per_layer_config[0].__getattribute__(key)  # read from a concrete layer
else:
    value = getattr(config, key)

def is_per_layer_attribute(config, key: str) -> bool:
    spec = getattr(config, "_heterogeneity_spec", None)
    return spec is not None and key in spec.per_layer_attributes

from transformers.integrations.heterogeneity.configuration_utils import AmbiguousGlobalPerLayerAttributeError

try:
    v = getattr(config, key)
except AmbiguousGlobalPerLayerAttributeError:
    v = getattr(config.per_layer_config[0], key)

When it happens

Trigger: Reading `config.num_key_value_heads` (any name registered as a per-layer attribute) on a config with a `_heterogeneity_spec` whose `per_layer_attributes` contains that key, while `config.allow_global_per_layer_attribute_access` is False (default). Triggers anywhere: user scripts, generic modeling code, `AutoModel` plumbing that introspects config attributes.

Common situations: Third-party or user code written for homogeneous models that assumes `config.<attr>` is authoritative; heterogeneous checkpoints (mixed GQA heads, mixed rope settings per layer) loaded through generic utils that read config attributes; serialization/inspection tooling that walks all config attributes.

Related errors

• Multiple valid text configs were found in the model config:
• `skip` must be an iterable of strings.
• `skip` must contain only strings.
• `per_layer_config` keys must be integer layer indices in the
• The following layers have the mutually exclusive `sliding_wi

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/15a91f01782f7cd1. Report an issue: GitHub.