sgl-project/sglang · error · RuntimeError

Cannot find CUTLASS headers required for JIT compilation. Pl

Error message

Cannot find CUTLASS headers required for JIT compilation. Please install flashinfer or deep_gemm with CUTLASS headers.

What it means

SGLang's JIT compiler needs CUTLASS headers to build CUTLASS-based kernels and resolves them from installed packages (flashinfer or deep_gemm ship CUTLASS headers). get_cutlass_include_paths() collects candidate include directories from those packages; if after deduplication no path exists, it raises this RuntimeError.

Source

Thrown at python/sglang/kernels/jit/utils/deps.py:137

                include_paths.append(str(path))

    deep_gemm_root = _find_package_root("deep_gemm")
    if deep_gemm_root is not None:
        candidate = deep_gemm_root / "include"
        if candidate.exists():
            include_paths.append(str(candidate))

    # De-duplicate while preserving order.
    unique_paths = []
    seen = set()
    for path in include_paths:
        if path in seen:
            continue
        seen.add(path)
        unique_paths.append(path)

    if not unique_paths:
        raise RuntimeError(
            "Cannot find CUTLASS headers required for JIT compilation. "
            "Please install flashinfer or deep_gemm with CUTLASS headers."
        )
    return unique_paths

View on GitHub (pinned to 0132848349)

Solutions

1. pip install flashinfer (its wheel bundles CUTLASS headers)
2. pip install deep_gemm as an alternative header source
3. Ensure the package's include directory actually contains cutlass/ (check site-packages/flashinfer/include/cutlass); reinstall if the install is broken
4. Manually place CUTLASS headers where the resolver looks, or update to an sglang version matching your installed flashinfer/deep_gemm

Example fix

# before: neither package present -> RuntimeError
pip install flashinfer-python -U --extra-index-url https://flashinfer.ai/whl/cu124
# after: headers resolvable
python -c "from sglang.kernels.jit.utils.deps import get_cutlass_include_paths; print(get_cutlass_include_paths())"

import importlib.util
if importlib.util.find_spec('flashinfer') is None and importlib.util.find_spec('deep_gemm') is None:
    raise SystemExit('Install flashinfer or deep_gemm (CUTLASS headers) before JIT compilation')

When it happens

Trigger: Compiling a JIT kernel that declares a cutlass dependency (calls get_cutlass_include_paths()) when neither flashinfer nor deep_gemm (nor any other searched package) is installed, or the installed versions do not ship include/ directories with CUTLASS headers.

Common situations: Minimal/lean install of sglang-srt without flashinfer or deep_gemm extras; a pip upgrade replaced flashinfer with a wheel variant that omits bundled CUTLASS headers; running in a slim container where headers were stripped to save image size.

Related errors

• Cannot find NVIDIA Math-DX (cuBLASDx) headers. Install the `
• Layer-sharded MLA HiCache backup with page_first layout requ
• Sync request failed: {error}
• Waiting for main node timeout!
• DeepGEMM Kernels compilation timeout.\n\nFeel free and pleas

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fc49230bcae012a7. Report an issue: GitHub.