{"record":{"id":"ea0d9f668c92c16b","repo":"huggingface/transformers","slug":"the-following-layers-have-the-mutually-exclusive","errorCode":null,"errorMessage":"The following layers have the mutually exclusive `sliding_window` and `attention_chunk_size` both defined: {problematic_indices}. To fix this, either remove a conflicting attribute from the global config,or set it to `None` in `per_layer_config` for the problematic layers.","messagePattern":"The following layers have the mutually exclusive `sliding_window` and `attention_chunk_size` both defined: (.+?)\\. To fix this, either remove a conflicting attribute from the global config,or set it to `None` in `per_layer_config` for the problematic layers\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/heterogeneity/configuration_utils.py","lineNumber":98,"sourceCode":"    config: PreTrainedConfig, per_layer_overrides: dict[int, dict[str, Any]]\n) -> None:\n    problematic_indices = []\n    for layer_idx in range(config.num_hidden_layers):\n        layer_overrides = per_layer_overrides.get(layer_idx, {})\n\n        sliding_window = layer_overrides.get(\n            \"sliding_window\", config._getattr_without_heterogeneous_validation(\"sliding_window\", None)\n        )\n        attention_chunk_size = layer_overrides.get(\n            \"attention_chunk_size\",\n            config._getattr_without_heterogeneous_validation(\"attention_chunk_size\", None),\n        )\n\n        if sliding_window is not None and attention_chunk_size is not None:\n            problematic_indices.append(layer_idx)\n\n    if problematic_indices:\n        raise ValueError(\n            f\"The following layers have the mutually exclusive `sliding_window` and `attention_chunk_size` both defined: \"\n            f\"{problematic_indices}. To fix this, either remove a conflicting attribute from the global config,\"\n            f\"or set it to `None` in `per_layer_config` for the problematic layers.\"\n        )\n\n\ndef _get_per_layer_attributes(per_layer_overrides: dict[int, dict[str, Any]]) -> set[str]:\n    per_layer_attributes: set[str] = set()\n    for layer_overrides in per_layer_overrides.values():\n        per_layer_attributes.update(layer_overrides)\n\n    per_layer_attributes.discard(\"skip\")\n    return per_layer_attributes\n\n\ndef _modify_config_and_create_heterogeneity_spec(\n    config: PreTrainedConfig, per_layer_overrides: dict[int, dict[str, Any]]\n) -> _HeterogeneitySpec:","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/heterogeneity/configuration_utils.py#L80-L116","documentation":"For heterogeneous configs, sliding_window and attention_chunk_size are mutually exclusive per layer. _validate_sliding_window_and_attention_chunk_size merges the per-layer override with the global config value (via _getattr_without_heterogeneous_validation) for every layer; any layer where both end up non-None is listed in the error, with instructions to remove one globally or explicitly null it out per layer.","triggerScenarios":"A global config that defines both sliding_window and attention_chunk_size (even if one is meant for a different layer subset) combined with per_layer_config that doesn't override either to None; or a per-layer override setting one while the global config supplies the other.","commonSituations":"Hybrid-attention models (e.g. Gemma-3/ModernBERT-style full+sliding layers or chunked-attention models) where both knobs appear in the config; inheriting a base config with sliding_window set and adding attention_chunk_size for a few layers without nulling the former.","solutions":["For the listed layers, set the conflicting attribute to None in per_layer_config: {layer: {\"sliding_window\": None}}","Or remove the unused attribute from the global config so only one is defined","Audit every problematic index listed in the message — each needs the conflict resolved"],"exampleFix":"# before: global config has sliding_window=1024 and attention_chunk_size=128\nconfig.per_layer_config = {0: {\"attention_chunk_size\": 128}}\n\n# after: null the global attr for that layer\nconfig.per_layer_config = {0: {\"attention_chunk_size\": 128, \"sliding_window\": None}}","handlingStrategy":"validation","validationCode":"def check_exclusive(config, per_layer):\n    bad = []\n    for i in range(config.num_hidden_layers):\n        ov = per_layer.get(i, {})\n        sw = ov.get(\"sliding_window\", getattr(config, \"sliding_window\", None))\n        acs = ov.get(\"attention_chunk_size\", getattr(config, \"attention_chunk_size\", None))\n        if sw is not None and acs is not None:\n            bad.append(i)\n    return bad\n\nassert not check_exclusive(config, per_layer_config), f\"conflicting layers: {check_exclusive(config, per_layer_config)}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["In hybrid models, explicitly set the non-applicable attention attribute to None per layer","Treat sliding_window and attention_chunk_size as mutually exclusive when authoring configs"],"tags":["heterogeneity","config","sliding-window","attention","mutually-exclusive"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}