huggingface/transformers · error · RuntimeError

replace_kernel_forward_from_hub requires `kernels` to be ins

Error message

replace_kernel_forward_from_hub requires `kernels` to be installed. Run `pip install kernels`.

What it means

`replace_kernel_forward_from_hub` is the decorator that swaps a layer's forward for a Hub kernel implementation. In the fallback block used when `kernels` is missing, the function itself is redefined to raise RuntimeError immediately, so any attempt to use the decorator (or call the function) reports the missing dependency.

Source

Thrown at src/transformers/integrations/hub_kernels.py:566

        def load(self):
            raise NotImplementedError("LayerRepository requires `kernels` to be installed. Run `pip install kernels.")

    class LocalLayerRepository:
        def __init__(self, *args, **kwargs):
            raise RuntimeError("LocalLayerRepository requires `kernels` to be installed. Run `pip install kernels`.")

        def load(self):
            raise NotImplementedError(
                "LocalLayerRepository requires `kernels` to be installed. Run `pip install kernels."
            )

    class FuncRepository:
        def __init__(self, *args, **kwargs):
            raise RuntimeError("FuncRepository requires `kernels` to be installed. Run `pip install kernels`.")

    def replace_kernel_forward_from_hub(*args, **kwargs):
        raise RuntimeError(
            "replace_kernel_forward_from_hub requires `kernels` to be installed. Run `pip install kernels`."
        )

    def register_kernel_mapping(*args, **kwargs):
        raise RuntimeError("register_kernel_mapping requires `kernels` to be installed. Run `pip install kernels`.")

    def register_kernel_mapping_transformers(*args, **kwargs):
        raise RuntimeError(
            "register_kernel_mapping_transformers requires `kernels` to be installed. Run `pip install kernels`."
        )


_HUB_KERNEL_MAPPING: dict[str, dict[str, str]] = {
    "finegrained-fp8": {"repo_id": "kernels-community/finegrained-fp8", "version": 4},
    "deep-gemm": {"repo_id": "kernels-community/deep-gemm", "version": 2},
    "sonic-moe": {"repo_id": "kernels-community/sonic-moe", "revision": "ep-support"},
}

View on GitHub (pinned to a597f97485)

Solutions

Install `kernels` before importing the kernelized model module.
Ensure the module only applies the decorator when `is_kernels_available()` (the normal transformers pattern guards these imports).
Use a non-kernelized variant of the model if kernels are not an option.

Example fix

# before (module top-level, breaks import without kernels)
@replace_kernel_forward_from_hub("kernels-community/fla")
class MyMambaLayer(MambaLayer): ...

# after
from transformers.utils.import_utils import is_kernels_available
if is_kernels_available():
    from transformers.integrations.hub_kernels import replace_kernel_forward_from_hub as _wrap
else:
    _wrap = lambda *a, **k: (lambda cls: cls)

@_wrap("kernels-community/fla")
class MyMambaLayer(MambaLayer): ...

Defensive patterns

Strategy: fallback

Validate before calling

from transformers.utils.import_utils import is_kernels_available
use_decorator = is_kernels_available()

Try / catch

try:
    import transformers.integrations.hub_kernels as hk
    assert callable(getattr(hk, "replace_kernel_forward_from_hub"))
except Exception:
    hk = None

Prevention

Install kernels before importing modules that use kernel decorators at class-definition time.
Wrap decorator application in a conditional so imports never hard-fail on slim environments.

When it happens

Trigger: Applying `@replace_kernel_forward_from_hub(...)` to a layer class, or calling it, in an environment without `kernels`. Typically triggered by importing kernelized model files that use the decorator at class-definition time.

Common situations: Importing a model whose code contains the decorator without the `kernels` extra installed; class-level decorators make this fire at import time, breaking the whole module import.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d9d377b6fc07d92a. Report an issue: GitHub.