sgl-project/sglang · error · RuntimeError

Failed to load HiCache native hash extension

Error message

Failed to load HiCache native hash extension

What it means

The torch.utils.cpp_extension.load call for the HiCache native hash extension (compiled C++ linked against -lcrypto) failed. The original exception is chained, so the real cause (missing compiler, missing OpenSSL dev headers, ninja, CUDA toolchain conflicts) is in __cause__.

Source

Thrown at python/sglang/srt/mem_cache/cpp_utils/native_hash.py:42

        )

    try:
        from torch.utils.cpp_extension import load

        abs_path = os.path.dirname(os.path.abspath(__file__))
        extra_cflags = ["-O3", "-std=c++17", "-DNDEBUG"]
        if _cpu_supports_avx2():
            extra_cflags.append("-mavx2")
        return load(
            name="hicache_hash_cpp",
            sources=[f"{abs_path}/hash_binding.cpp"],
            extra_cflags=extra_cflags,
            extra_ldflags=["-lcrypto"],
            with_cuda=False,
            verbose=False,
        )
    except Exception as exc:
        raise RuntimeError("Failed to load HiCache native hash extension") from exc


def _native_hash_input(token_ids: Any) -> tuple[array, int, int, bool]:
    raw_token_ids = getattr(token_ids, "raw_token_ids", None)
    raw = (
        raw_token_ids()
        if raw_token_ids is not None
        else getattr(token_ids, "token_ids", token_ids)
    )

    logical_len = len(token_ids)
    is_bigram = getattr(token_ids, "is_bigram", False)

    if isinstance(raw, array) and raw.typecode in ("I", "q", "Q", "L"):
        if is_bigram and logical_len > 0 and len(raw) < logical_len + 1:
            raise ValueError("bigram token buffer is shorter than logical length")
        return raw, logical_len, 2 if is_bigram else 1, is_bigram

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the chained __cause__ exception for the actual compiler/linker error
2. Install build prerequisites: apt-get install -y build-essential libssl-dev ninja-build (or OS equivalent)
3. Retry after cleaning torch extension caches (~/.cache/torch_extensions)
4. Use the pure-Python hash path as a fallback if the native hash is optional for you

Example fix

# before
h = get_native_hash()
# after
try:
    h = get_native_hash()
except RuntimeError as e:
    logger.warning("native hash unavailable (%s): %s", e, e.__cause__)
    h = None  # fall back to Python hash

import shutil
shutil.which("gcc") or shutil.which("clang"); shutil.which("ninja")
# and check libcrypto headers exist before first run

try:\n    h = get_native_hash()\nexcept RuntimeError as e:\n    log.warning("native hash load failed: %s (cause: %s)", e, e.__cause__)\n    h = None

When it happens

Trigger: get_native_hash() triggering JIT compilation where clang/gcc, libcrypto (OpenSSL dev package), or ninja is missing, or a compiler/toolchain version error occurs; any exception inside load() is re-raised as this RuntimeError.

Common situations: Slim Docker images without build-essential or libssl-dev; first-run JIT compile in an environment without ninja; incompatible PyTorch/CUDA versions breaking cpp_extension; no network/cache for extension builds.

Related errors

• Unsupported content type ${header.content_type}
• Can not import FA3 in sgl_kernel. Please check your installa
• st attn not supported
• HiCache native hash is only supported on little-endian Linux
• failed to query the Rust toolchain with `{command} {' '.join

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cd3931ff9c30cc45. Report an issue: GitHub.