huggingface/transformers · error · ValueError

The `{layer_types}` entries must be in {allowed_types} but g

Error message

The `{layer_types}` entries must be in {allowed_types} but got {layers}

What it means

ValueError from layer-type validation when any entry of layer_types or mlp_layer_types is not in the corresponding allowed set (ALLOWED_ATTN_LAYER_TYPES / ALLOWED_MLP_LAYER_TYPES). Custom-code configs first get legacy names remapped; anything still unknown after remapping is rejected. It guards against silently building layers the modeling code cannot instantiate.

Source

Thrown at src/transformers/configuration_utils.py:539

                        f"and {vocab_size - 1}), got {value}. This may result in unexpected behavior."
                    )

    def validate_layer_type(self):
        """Check that `mlp_layer_types` and `layer_types` is correctly defined."""
        for allowed_types, layer_types in zip(
            [ALLOWED_ATTN_LAYER_TYPES, ALLOWED_MLP_LAYER_TYPES], ["layer_types", "mlp_layer_types"]
        ):
            layers = getattr(self, layer_types, None)
            if not (layers is not None and hasattr(self, "num_hidden_layers")):
                return
            if self.is_custom_code():
                # Custom code may have legacy layer types that need to be remapped
                if (remapped := remap_legacy_layer_types(layers)) != layers:
                    # Only try setattr if layers changed in case layer_types is a read-only property
                    setattr(self, layer_types, remapped)
                layers = remapped
            if not all(layer_type in allowed_types for layer_type in layers):
                raise ValueError(f"The `{layer_types}` entries must be in {allowed_types} but got {layers}")
            elif self.num_hidden_layers is not None and self.num_hidden_layers != len(layers):
                raise ValueError(
                    f"`num_hidden_layers` ({self.num_hidden_layers}) must be equal to the number of `{layer_types}` "
                    f"({len(layers)})"
                )

    @property
    def rope_scaling(self):
        return self.rope_parameters

    @rope_scaling.setter
    def rope_scaling(self, value):
        self.rope_parameters = value

    def save_pretrained(self, save_directory: str | os.PathLike, push_to_hub: bool = False, **kwargs):
        """
        Save a configuration object to the directory `save_directory`, so that it can be re-loaded using the
        [`~PreTrainedConfig.from_pretrained`] class method.

View on GitHub (pinned to a597f97485)

Solutions

1. Replace each invalid entry (the message lists the offending list and the allowed set) with a valid type such as 'self_attn', 'cross_attn', or the documented MLP types
2. Print ALLOWED_ATTN_LAYER_TYPES / ALLOWED_MLP_LAYER_TYPES from your transformers version for the exact vocabulary
3. For custom-code models, ensure is_custom_code() is true so legacy remapping applies, or update the names

Example fix

# before
cfg.layer_types = ['self_attn', 'banana_attn']
# after
cfg.layer_types = ['self_attn', 'cross_attn']

from transformers.configuration_utils import ALLOWED_ATTN_LAYER_TYPES, ALLOWED_MLP_LAYER_TYPES
assert all(t in ALLOWED_ATTN_LAYER_TYPES for t in layer_types), f'bad types: {set(layer_types) - set(ALLOWED_ATTN_LAYER_TYPES)}'

def layer_types_valid(layer_types: list[str], allowed: set[str]) -> bool:
    return all(t in allowed for t in layer_types)

try:
    cfg.validate_layer_types()
except ValueError as e:
    if 'must be in' in str(e):
        raise ValueError(f'unknown layer type; allowed: {ALLOWED_ATTN_LAYER_TYPES}') from e
    raise

When it happens

Trigger: MyConfig(layer_types=['self_attn', 'banana_attn']) where 'banana_attn' is not an allowed attention layer type; passing vendor-specific layer names to a stock transformers build; stale names in a remoted config that remap_legacy_layer_types does not cover.

Common situations: Configs authored against forks that extended the layer-type vocabulary; version skew where allowed types were renamed; hand-edited config.json layer lists.

Related errors

• `num_hidden_layers` ({self.num_hidden_layers}) must be equal
• The embed_dim ({self.embed_dim}) is not a multiple of the nu
• The `layer_types` entries must be in {ALLOWED_LAYER_TYPES}
• `num_hidden_layers` ({num_hidden_layers}) must be equal to t
• Layer type '{layer_idx}' not found in config.layer_types: {l

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/30384339820b3a69. Report an issue: GitHub.