sgl-project/sglang · error · RuntimeError

HiCache native hash is only supported on little-endian Linux

Error message

HiCache native hash is only supported on little-endian Linux

What it means

The HiCache native hash extension only builds/runs on little-endian Linux. _load_native_hash_module checks sys.byteorder and sys.platform up front and raises this RuntimeError elsewhere (e.g. macOS, big-endian s390x/PowerPC) before attempting the torch cpp_extension load.

Source

Thrown at python/sglang/srt/mem_cache/cpp_utils/native_hash.py:22

from array import array
from functools import lru_cache
from typing import Any, Optional


def _cpu_supports_avx2() -> bool:
    if platform.machine().lower() not in ("x86_64", "amd64"):
        return False
    try:
        with open("/proc/cpuinfo", "r", encoding="utf-8", errors="ignore") as f:
            return "avx2" in f.read().lower()
    except OSError:
        return False


@lru_cache(maxsize=1)
def _load_native_hash_module() -> Any:
    if sys.byteorder != "little" or not sys.platform.startswith("linux"):
        raise RuntimeError(
            "HiCache native hash is only supported on little-endian Linux"
        )

    try:
        from torch.utils.cpp_extension import load

        abs_path = os.path.dirname(os.path.abspath(__file__))
        extra_cflags = ["-O3", "-std=c++17", "-DNDEBUG"]
        if _cpu_supports_avx2():
            extra_cflags.append("-mavx2")
        return load(
            name="hicache_hash_cpp",
            sources=[f"{abs_path}/hash_binding.cpp"],
            extra_cflags=extra_cflags,
            extra_ldflags=["-lcrypto"],
            with_cuda=False,
            verbose=False,
        )

View on GitHub (pinned to 0132848349)

Solutions

Run the workload on little-endian Linux (x86_64 or aarch64 Linux)
Use the pure-Python hash fallback instead of get_native_hash if your code path allows it
For local development, use Docker with a linux/amd64 or linux/arm64 image

Example fix

# before
from sglang.srt.mem_cache.cpp_utils.native_hash import get_native_hash
h = get_native_hash()
# after
import sys
if sys.byteorder == "little" and sys.platform.startswith("linux"):
    h = get_native_hash()
else:
    h = None  # use pure-Python hash path

Defensive patterns

Strategy: type-guard

Validate before calling

import sys
_NATIVE_HASH_OK = sys.byteorder == "little" and sys.platform.startswith("linux")
if not _NATIVE_HASH_OK:
    use_python_hash_fallback()

Type guard

def native_hash_supported() -> bool:
    import sys
    return sys.byteorder == "little" and sys.platform.startswith("linux")

Try / catch

try:\n    h = get_native_hash()\nexcept RuntimeError:\n    h = None  # fallback to pure-Python hashing

Prevention

Develop inside linux containers to match production
Feature-detect platform before first native hash use
Keep a pure-Python hash fallback wired in your code

When it happens

Trigger: Calling get_native_hash() on big-endian systems or non-Linux platforms (macOS ARM/Intel, Windows, FreeBSD); the platform guard fires before any extension compilation.

Common situations: Developing or unit-testing SGLang on a Mac or Windows workstation; running in an unusual container reporting a non-linux platform; big-endian enterprise hardware.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/94afcee3260d9c83. Report an issue: GitHub.