huggingface/transformers · error · ImportError

finegrained-fp8 kernel is missing required symbols: {', '.jo

Error message

finegrained-fp8 kernel is missing required symbols: {', '.join(missing)}. {_MISSING_KERNELS_MESSAGE}

What it means

Raised after the finegrained-fp8 kernel module loads but lacks one or more of the required symbols matmul_2d, matmul_batched, or matmul_grouped. Transformers pins an implicit contract on the kernel's API surface; an older (or differently versioned) kernels-community/finegrained-fp8 build that predates one of these entry points fails the getattr(None) check and this ImportError lists exactly which symbols are missing.

Source

Thrown at src/transformers/integrations/finegrained_fp8.py:132

            "Failed to load the finegrained-fp8 kernel — check that `kernels-community/finegrained-fp8` "
            "has a build matching the current torch/CUDA."
        )

    matmul = getattr(kernel, "matmul_2d", None)
    batched_matmul = getattr(kernel, "matmul_batched", None)
    grouped_matmul = getattr(kernel, "matmul_grouped", None)

    missing = [
        name
        for name, attr in [
            ("matmul_2d", matmul),
            ("matmul_batched", batched_matmul),
            ("matmul_grouped", grouped_matmul),
        ]
        if attr is None
    ]
    if missing:
        raise ImportError(
            f"finegrained-fp8 kernel is missing required symbols: {', '.join(missing)}. {_MISSING_KERNELS_MESSAGE}"
        )

    _FINEGRAINED_FP8 = FineGrainedFP8(
        matmul=matmul,
        batched_matmul=batched_matmul,
        grouped_matmul=grouped_matmul,
    )


def load_finegrained_fp8_kernel() -> FineGrainedFP8:
    _load_finegrained_fp8_kernel()
    return _FINEGRAINED_FP8


def _cdiv(a: int, b: int) -> int:
    """Ceiling division."""
    return (a + b - 1) // b

View on GitHub (pinned to a597f97485)

Solutions

1. Clear the cached kernel build (HF/kernels cache, e.g. ~/.cache/kernels or the kernels cache dir) so a fresh build is fetched
2. Upgrade transformers and the kernels package together so the required symbol set matches: pip install -U transformers kernels
3. Verify what the loaded kernel exports: python -c "from kernels import lazy_load_kernel; k = lazy_load_kernel('finegrained-fp8'); print([a for a in dir(k) if 'matmul' in a])"

Example fix

# before
rm -rf ~/.cache/kernels  # stale build missing matmul_grouped
load_finegrained_fp8_kernel()  # ImportError: missing required symbols: matmul_grouped

# after: fetch a fresh build matching current transformers
rm -rf ~/.cache/kernels && pip install -U kernels transformers
load_finegrained_fp8_kernel()

from kernels import lazy_load_kernel
k = lazy_load_kernel("finegrained-fp8")
required = {"matmul_2d", "matmul_batched", "matmul_grouped"}
missing = required - {a for a in required if getattr(k, a, None) is not None}
assert not missing, f"stale kernel build, missing {missing}; clear kernels cache"

try:
    load_finegrained_fp8_kernel()
except ImportError as e:
    if "missing required symbols" in str(e):
        # stale cached build — clear and refetch once, then fail hard if still broken
        import shutil, pathlib
        shutil.rmtree(pathlib.Path.home() / ".cache" / "kernels", ignore_errors=True)
    raise

When it happens

Trigger: load_finegrained_fp8_kernel() when a stale cached build of finegrained-fp8 is present (e.g. cached from an older kernels release) or the published kernel version does not yet export matmul_grouped / matmul_batched.

Common situations: A kernels cache directory persisted across a transformers upgrade that started requiring a new symbol; a pinned old commit of the kernel in a lockfile; partially-populated HF cache after an interrupted download.

Related errors

• finegrained-fp8 kernel unavailable: {_MISSING_KERNELS_MESSAG
• Failed to load the finegrained-fp8 kernel — check that `kern
• You need to install optimum-quanto in order to use KV cache
• You need to install `HQQ` in order to use KV cache quantizat
• `axis_value` for `HQQ` backend has to be one of [`0`, `1`] b

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d48a6dff661dc34d. Report an issue: GitHub.