sgl-project/sglang · error · ValueError

--hicache-host-memory-mode buffer_only is only implemented f

Error message

--hicache-host-memory-mode buffer_only is only implemented for the unified radix tree; this model selected {type(cache).__name__}.

What it means

create_tree_cache validates that --hicache-host-memory-mode buffer_only is only implemented for UnifiedRadixCache. If the chosen backend/model combination produced a different cache class (e.g. the C++ tree or a custom registered backend), startup fails with this ValueError.

Source

Thrown at python/sglang/srt/mem_cache/registry.py:223

            raise ValueError(
                f"--radix-cache-backend={name!r} is not registered. "
                f"Registered backends: {registered_radix_cache_backends()}. "
                "External backends must call register_radix_cache_backend(...) at import time."
            )
        cache = factory(ctx)
        source = f"registered({name!r})"
    else:
        cache = default_radix_cache_factory(ctx)
        source = "default"

    if (
        get_memory().enable_hierarchical_cache
        and ctx.server_args.hicache_host_memory_mode == "buffer_only"
    ):
        from sglang.srt.mem_cache.unified_radix_cache import UnifiedRadixCache

        if not isinstance(cache, UnifiedRadixCache):
            raise ValueError(
                "--hicache-host-memory-mode buffer_only is only implemented for "
                f"the unified radix tree; this model selected {type(cache).__name__}."
            )

    if ctx.server_args.enable_session_radix_cache and not getattr(
        cache, "enable_session_radix_cache", False
    ):
        raise ValueError(
            "--enable-session-radix-cache requires UnifiedRadixCache, but "
            f"tree_cache is {type(cache).__name__}. Drop the flag or the "
            "option that selected another tree cache for this model."
        )

    hicache_attached = cache.cache_controller is not None
    streaming_wrapped = False
    if (
        ctx.server_args.enable_streaming_session
        and not cache.supports_streaming_session()

View on GitHub (pinned to 0132848349)

Solutions

  1. Drop --hicache-host-memory-mode buffer_only, or use the default unified radix tree
  2. Switch --radix-cache-backend to the unified/default backend
  3. If you wrote a custom backend, either subclass/wrap UnifiedRadixCache or implement buffer_only support

Example fix

# before
--hicache-host-memory-mode buffer_only --radix-cache-backend cpp
# after
--hicache-host-memory-mode buffer_only  # default unified tree
Defensive patterns

Strategy: validation

Validate before calling

if server_args.hicache_host_memory_mode == "buffer_only" and server_args.radix_cache_backend not in (None, ""):
    raise ValueError("buffer_only requires the unified radix tree")

Prevention

When it happens

Trigger: Launching with --hicache-host-memory-mode buffer_only together with --radix-cache-backend selecting a non-unified cache (cpp backend, custom plugin), or a model/config whose default factory returns a non-UnifiedRadixCache.

Common situations: Mixing experimental flags: enabling buffer_only host mode while testing the C++ radix tree or a third-party cache backend.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/356b1ff6581aea01. Report an issue: GitHub.