huggingface/transformers · error · ValueError

Expected exactly one kernel repo regardless of device/mode s

Error message

Expected exactly one kernel repo regardless of device/mode specificity, got {hub_repo}

What it means

In a `KernelConfig.kernel_mapping`, the value for each layer name may be a plain repo string, a (repo, metadata) tuple, or a dict keyed by device/mode — but after normalization the dict must contain exactly one repo entry, because `register_kernel_replacements_and_fusions` resolves one hub repo per layer regardless of device or training/inference mode. A dict with zero or 2+ entries is rejected with this ValueError.

Source

Thrown at src/transformers/integrations/hub_kernels.py:873

) -> None:
    if not hasattr(cls, "config_class") or not hasattr(cls.config_class, "model_type"):
        raise ValueError(f"Model {cls.__name__} has no config_class or model_type.")
    model_type = cls.config_class.model_type

    patch_mapping: dict[str, type] = {}
    new_mapping: dict = {}

    # We might need to instantiate the model on meta device.
    # We do it lazily, only if we encounter a fused kernel.
    meta_model = None

    for layer_name, hub_repo in kernel_config.kernel_mapping.items():
        if isinstance(hub_repo, (str, tuple)):
            hub_repo = {None: hub_repo}

        if isinstance(hub_repo, dict):
            if len(hub_repo.values()) != 1:
                raise ValueError(
                    f"Expected exactly one kernel repo regardless of device/mode specificity, got {hub_repo}"
                )
        else:
            raise ValueError(f"Invalid hub repo {hub_repo!r} for layer {layer_name!r}")

        hub_repo = next(iter(hub_repo.values()))

        # Infer metadata (revision/version/trust_remote_code)
        if isinstance(hub_repo, tuple):
            repo_str, metadata = hub_repo

            revision = metadata.get("revision", None)
            version = metadata.get("version", None)
            trust_remote_code = metadata.get("trust_remote_code", False) or ALLOW_ALL_KERNELS
            metadata = {"version": version} if version is not None else {"revision": revision}
            metadata |= {"trust_remote_code": trust_remote_code}

            final_repo = (repo_str, metadata)

View on GitHub (pinned to a597f97485)

Solutions

1. Give each layer exactly one repo, ideally as a plain string: `{"LlamaDecoderLayer": "kernels-community/llama-blk"}`.
2. If you used a dict form, keep one key only, e.g. `{None: "repo"}` or one device key.
3. Remove empty `{}` entries from the mapping.
4. Do device-based selection in your own code before building the KernelConfig.

Example fix

# before
kernel_mapping = {
    "LlamaDecoderLayer": {"cuda": "kernels-community/llama-blk", "rocm": "kernels-community/llama-blk-rocm"},
}
# ValueError: Expected exactly one kernel repo ...

# after
kernel_mapping = {
    "LlamaDecoderLayer": "kernels-community/llama-blk",  # pick one repo per layer
}

def normalize_kernel_mapping(mapping):
    out = {}
    for layer, repo in mapping.items():
        if isinstance(repo, dict) and len(repo) != 1:
            raise ValueError(f"layer {layer!r} must map to exactly one kernel repo, got {repo}")
        out[layer] = next(iter(repo.values())) if isinstance(repo, dict) else repo
    return out

kernel_mapping = normalize_kernel_mapping(kernel_mapping)

def is_valid_kernel_mapping(mapping: dict) -> bool:
    return all(
        isinstance(v, (str, tuple)) or (isinstance(v, dict) and len(v) == 1)
        for v in mapping.values()
    )

try:
    register_kernel_replacements_and_fusions(cls, config, kernel_config)
except ValueError as e:
    if "exactly one kernel repo" in str(e):
        for layer, repo in kernel_config.kernel_mapping.items():
            if isinstance(repo, dict) and len(repo) != 1:
                kernel_config.kernel_mapping[layer] = next(iter(repo.values())) or None
        register_kernel_replacements_and_fusions(cls, config, kernel_config)
    else:
        raise

When it happens

Trigger: Hand-writing a kernel mapping like `{"model.layers.0.mlp": {"cuda": "repo_a", "cpu": "repo_b"}}` (two entries), or `{"layer": {}}` (zero entries), and passing it as `KernelConfig(kernel_mapping=...)` to a kernelized `from_pretrained` / `register_kernel_replacements_and_fusions`. Plain string values or a single-key dict (`{None: "repo"}`) are accepted.

Common situations: Users assuming per-device kernel selection is supported and authoring multi-entry dicts; merging/combining kernel configs that accidentally produce multiple keys; empty placeholder entries left in a config file.

Related errors

• register_kernel_mapping requires `kernels` to be installed.
• register_kernel_mapping_transformers requires `kernels` to b
• No baseline with name '{name}' in {RESULTS_DIR}
• Tensor parallelism was requested, but WORLD_SIZE is not set
• Unknown fusion type: {fusion_name}

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/7a1a60cd2faa338c. Report an issue: GitHub.