sgl-project/sglang · error · ValueError

hicache_host_memory_mode must be 'cache' or 'buffer_only', g

Error message

hicache_host_memory_mode must be 'cache' or 'buffer_only', got {cfg.hicache_host_memory_mode!r}

What it means

SGLang's HiCache host memory accepts exactly two modes: 'cache' (host memory as a caching tier) and 'buffer_only' (host memory as a plain staging buffer). _validate_hicache_host_memory_mode rejects any other value of --hicache-host-memory-mode before the server starts.

Source

Thrown at python/sglang/srt/server_args.py:8297

            )
        ):
            return

        self._validate_hicache_host_memory_mode()

        # Step 1: Initial layout-io compatibility normalization.
        self._resolve_layout_io_compatibility()

        # Step 2: Storage-layout normalization without changing io backend.
        self._resolve_storage_layout_compatibility()

        # Step 3: DCP compatibility for the L2 (device<->host) path.
        self._resolve_hicache_dcp_compatibility()

    def _validate_hicache_host_memory_mode(self):
        cfg = resolving_view(self)
        if cfg.hicache_host_memory_mode not in ("cache", "buffer_only"):
            raise ValueError(
                "hicache_host_memory_mode must be 'cache' or 'buffer_only', "
                f"got {cfg.hicache_host_memory_mode!r}"
            )

        # Both modes are defaulted upstream (a decode server resolves the
        # ratio later, in kv_cache_builder), so this fires only if that
        # defaulting regresses -- never build an unsized host pool.
        if (
            cfg.hicache_size <= 0
            and cfg.hicache_ratio is None
            and cfg.disaggregation_mode != "decode"
        ):
            raise ValueError(
                f"--hicache-host-memory-mode {cfg.hicache_host_memory_mode} "
                "requires a host pool size: pass --hicache-size or "
                "--hicache-ratio."
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Correct the value to exactly 'cache' or 'buffer_only'
2. Remove the flag to use the default mode
3. Check the current --help output for the accepted enum values of your SGLang version

Example fix

# before
--hicache-host-memory-mode buffer
# after
--hicache-host-memory-mode buffer_only

VALID_HICACHE_HOST_MODES = {"cache", "buffer_only"}
mode = parsed.get("hicache_host_memory_mode")
if mode is not None and mode not in VALID_HICACHE_HOST_MODES:
    raise SystemExit(f"hicache_host_memory_mode must be one of {sorted(VALID_HICACHE_HOST_MODES)}")

def is_valid_hicache_host_mode(mode: str) -> bool:
    return mode in {"cache", "buffer_only"}

When it happens

Trigger: Passing --hicache-host-memory-mode with a typo'd or unsupported value (anything outside {"cache","buffer_only"}), e.g. buffer, host_cache, or none.

Common situations: Typos in launch scripts, stale flags from older versions where the option did not exist or had different names, or copying configs between SGLang versions.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Not enough host memory for V4 paged pool {pool_name}. Reques
• The arguments enable-hierarchical-cache and disable-radix-ca
• lora_alpha must be a positive integer
• DeepSeekV4 only supports interleave CP strategy, got {cfg.cp
• HiSparse requires one of {HISPARSE_KV_CACHE_DTYPES} KV cache

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6d022b8cf1cf3993. Report an issue: GitHub.