sgl-project/sglang · error · ValueError

--hicache-host-memory-mode buffer_only does not support side

Error message

--hicache-host-memory-mode buffer_only does not support sidecar storage pools (DeepSeek-V4 compressed regions).

What it means

Raised during HiCache initialization when --hicache-host-memory-mode buffer_only is combined with a model that uses sidecar storage pools (DeepSeek-V4 compressed regions). Buffer-only mode has no per-pool staging path for sidecar pools, so the configuration is rejected upfront rather than silently misbehaving at runtime.

Source

Thrown at python/sglang/srt/mem_cache/buffer_mode/pipeline.py:159

    """Drop one content ref per page hash (at storage-ack)."""
    for h in hash_values:
        n = refs.get(h, 0) - 1
        if n <= 0:
            refs.pop(h, None)
        else:
            refs[h] = n


def validate_buffer_only_stack(
    sidecar_pool_specs: list, swa_component: Optional[SWAComponent]
) -> None:
    """Post-assembly buffer-mode fences.

    Sidecar pools (DSv4 compressed regions) and unified_kv SWA (device-only
    ring, never offloaded) have no per-pool staging path yet.
    """
    if sidecar_pool_specs:
        raise ValueError(
            "--hicache-host-memory-mode buffer_only does not support "
            "sidecar storage pools (DeepSeek-V4 compressed regions)."
        )
    swa = swa_component
    if swa is not None and swa._swa_kv_pool_host is None:
        # Only reachable on SWA models with the unified_kv layout (SWA as
        # a device-only ring): without a host pool the window can neither
        # stage for writes nor fetch for load-backs.
        raise ValueError(
            "--hicache-host-memory-mode buffer_only on SWA models "
            "requires an SWA host staging pool; the unified_kv layout "
            "keeps SWA as a device-only ring."
        )
    if swa is not None and swa._swa_kv_pool_host is not None:
        # Below two windows the pool cannot hold a staging write AND the
        # loads-priority reserve (_aux_loads_margin floors at one
        # window), so every window-carrying intent would be dropped as
        # oversize and SWA storage coverage would silently be zero.

View on GitHub (pinned to 0132848349)

Solutions

Remove --hicache-host-memory-mode buffer_only (or drop the flag) for DeepSeek-V4 models with compressed sidecar regions
Switch to a supported hicache host memory mode (e.g. the default/preload mode) that supports sidecar pools
If you don't need DSv4 compressed regions, load a model without sidecar pools before using buffer_only

Example fix

# before
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V4 --hicache-host-memory-mode buffer_only
# after
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V4

Defensive patterns

Strategy: validation

Validate before calling

sidecar = getattr(model_config_memory_specs, "sidecar_pool_specs", None)
if server_args.hicache_host_memory_mode == "buffer_only" and sidecar:
    raise SystemExit("buffer_only unsupported with sidecar pools; drop the flag")

Prevention

Check hicache mode compatibility flags in a launch wrapper script for new model families
Read release notes when enabling DSv4 compressed regions
Fail fast at startup with explicit arg validation rather than relying on init_hicache

When it happens

Trigger: Launching a server with --hicache-host-memory-mode buffer_only on a DeepSeek-V4 model whose memory pool specs include sidecar pools (compressed regions); init_hicache calls validate_buffer_only_stack with non-empty sidecar_pool_specs.

Common situations: Enabling buffer_only host memory mode to cut host RAM usage on a DSv4 compressed-KV model; upgrading to a model with sidecar pools without changing the hicache flags.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e61e9c6f4edc56d8. Report an issue: GitHub.