{"record":{"id":"99a919e7ad7c93d8","repo":"huggingface/transformers","slug":"finegrained-fp8-kernel-unavailable-missing-kern","errorCode":null,"errorMessage":"finegrained-fp8 kernel unavailable: {_MISSING_KERNELS_MESSAGE}","messagePattern":"finegrained-fp8 kernel unavailable: (.+?)","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/finegrained_fp8.py","lineNumber":109,"sourceCode":"    Load the finegrained-fp8 Triton kernel once into the `_FINEGRAINED_FP8` module global.\n\n    `@allow_in_graph` makes `torch.compile` treat the untraceable hub download + dynamic import as a\n    single opaque node instead of tracing into it; it returns `None` (proxyable) and populates the\n    global, which `load_finegrained_fp8_kernel` then returns.\n\n    Under NO circumstances may this function return a value: an `@allow_in_graph` fx node's\n    return must be proxyable, and returning the bundle (e.g. from the warm-cache\n    short-circuit) breaks torch.compile with `Unsupported: torch.* op returned non-Tensor`.\n\n    Raises `ImportError` if the `kernels` package is missing, or the kernel or required\n    symbols cannot be found.\n    \"\"\"\n    global _FINEGRAINED_FP8\n    if _FINEGRAINED_FP8 is not None:\n        return\n\n    if not is_kernels_available():\n        raise ImportError(f\"finegrained-fp8 kernel unavailable: {_MISSING_KERNELS_MESSAGE}\")\n\n    kernel = lazy_load_kernel(\"finegrained-fp8\")\n    if kernel is None:\n        raise ImportError(\n            \"Failed to load the finegrained-fp8 kernel — check that `kernels-community/finegrained-fp8` \"\n            \"has a build matching the current torch/CUDA.\"\n        )\n\n    matmul = getattr(kernel, \"matmul_2d\", None)\n    batched_matmul = getattr(kernel, \"matmul_batched\", None)\n    grouped_matmul = getattr(kernel, \"matmul_grouped\", None)\n\n    missing = [\n        name\n        for name, attr in [\n            (\"matmul_2d\", matmul),\n            (\"matmul_batched\", batched_matmul),\n            (\"matmul_grouped\", grouped_matmul),","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/finegrained_fp8.py#L91-L127","documentation":"Raised by _load_finegrained_fp8_kernel() in src/transformers/integrations/finegrained_fp8.py when the lazy finegrained-fp8 kernel loader runs and is_kernels_available() returns False. Transformers only vendors the integration glue; the actual Triton/CUDA kernels live in the external 'kernels' package (kernels-community/finegrained-fp8 on Hugging Face). Without that package installed the integration cannot construct the FineGrainedFP8 bundle and refuses to continue.","triggerScenarios":"Calling load_finegrained_fp8_kernel(), or any code path that reaches it (e.g. _apply_finegrained_fp8 / enabling an fp8 recipe on a model) after importing transformers, on an environment where `import kernels` fails or the kernels package was never installed.","commonSituations":"User enables finegrained FP8 quantization (e.g. via a quantization_config selecting finegrained-fp8) in a fresh venv or a Docker image that only pip-installed transformers[torch]. CI images that strip optional dependencies also hit this.","solutions":["pip install kernels (the HF kernels runtime, https://github.com/huggingface/kernels)","If kernels is installed but broken, verify with `python -c \"import kernels\"` and reinstall it","If you did not intend to use finegrained-fp8, remove or change the quantization config that triggers this code path"],"exampleFix":"# before\nmodel = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=FineGrainedFP8Config())\n# ImportError: finegrained-fp8 kernel unavailable\n\n# after\npip install kernels\nmodel = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=FineGrainedFP8Config())","handlingStrategy":"validation","validationCode":"from transformers.utils import is_kernels_available\n\nif not is_kernels_available():\n    raise SystemExit(\"pip install kernels before enabling finegrained-fp8\")\nmodel = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=finegrained_fp8_cfg)","typeGuard":null,"tryCatchPattern":"try:\n    from transformers.integrations.finegrained_fp8 import load_finegrained_fp8_kernel\n    load_finegrained_fp8_kernel()\nexcept ImportError as e:\n    if \"kernels\" in str(e):\n        print(\"Missing dependency: pip install kernels\")\n    raise","preventionTips":["Declare `kernels` in your project's dependencies whenever finegrained-fp8 quantization is enabled","Add a startup check: from transformers.utils import is_kernels_available; assert is_kernels_available()","Pin transformers and kernels versions together in your lockfile so the kernel API contract holds"],"tags":["fp8","quantization","kernels","import-error","optional-dependency"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}