sgl-project/sglang · error · TypeError

Dynamic HiCache sidecars require HostPoolGroup.

Error message

Dynamic HiCache sidecars require HostPoolGroup.

What it means

HybridCacheController.register_host_pool_entry registers an extra host memory pool ('sidecar') for dynamic HiCache, but this only works when the controller's host memory is organized as a HostPoolGroup (multiple named pools). If mem_pool_host is a single flat pool (not a HostPoolGroup instance), adding named entries is meaningless and a TypeError is raised.

Source

Thrown at python/sglang/srt/mem_cache/hybrid_cache/hybrid_cache_controller.py:173

        storage_backend: str,
        prefetch_threshold: int = 256,
        model_name: Optional[str] = None,
        storage_backend_extra_config: Optional[dict] = None,
        host_pools: Optional[list[PoolEntry]] = None,
    ):
        super().attach_storage_backend(
            storage_backend=storage_backend,
            prefetch_threshold=prefetch_threshold,
            model_name=model_name,
            storage_backend_extra_config=storage_backend_extra_config,
        )

        for entry in host_pools or []:
            self.storage_backend.register_mem_host_pool_v2(entry.host_pool, entry.name)

    def register_host_pool_entry(self, entry: PoolEntry) -> None:
        if not isinstance(self.mem_pool_host, HostPoolGroup):
            raise TypeError("Dynamic HiCache sidecars require HostPoolGroup.")
        self.mem_pool_host.add_entry(entry)
        if not entry.is_primary_index_anchor:
            self.extra_host_mem_release_queues.setdefault(entry.name, Queue())
        if self.enable_storage and self.storage_backend is not None:
            self.storage_backend.register_mem_host_pool_v2(entry.host_pool, entry.name)

    @staticmethod
    def parse_storage_backend_extra_config(
        storage_backend_extra_config: Optional[str],
    ) -> tuple[dict, int, float, float, bool]:
        extra_config = {}
        if storage_backend_extra_config:
            if storage_backend_extra_config.startswith("@"):
                path = storage_backend_extra_config[1:]
                ext = os.path.splitext(path)[1].lower()
                with open(path, "rb" if ext == ".toml" else "r") as f:
                    if ext == ".json":
                        extra_config = json.load(f)

View on GitHub (pinned to 0132848349)

Solutions

1. Initialize the controller with a HostPoolGroup as mem_pool_host (enable the dynamic sidecar / host pool group option in server args) before registering sidecar entries
2. Guard the call: check isinstance(controller.mem_pool_host, HostPoolGroup) and skip/log otherwise
3. If you don't need dynamic sidecars, remove the register_sidecar_pool call

Example fix

# before
controller.register_host_pool_entry(entry)  # mem_pool_host is a flat pool

# after
from sglang.srt.mem_cache.hybrid_allocator import HostPoolGroup
assert isinstance(controller.mem_pool_host, HostPoolGroup), "enable HostPoolGroup before adding sidecars"
controller.register_host_pool_entry(entry)

from sglang.srt.mem_cache.hybrid_allocator import HostPoolGroup
if not isinstance(controller.mem_pool_host, HostPoolGroup):
    raise RuntimeError("construct controller with HostPoolGroup before registering sidecars")

def can_register_sidecar(controller) -> bool:
    return isinstance(getattr(controller, "mem_pool_host", None), HostPoolGroup)

try:
    controller.register_host_pool_entry(entry)
except TypeError as e:
    if "HostPoolGroup" in str(e):
        logger.warning("sidecar registration skipped: HostPoolGroup not enabled")
    else:
        raise

When it happens

Trigger: Calling register_host_pool_entry / register_sidecar_pool on a HybridCacheController that was constructed with a plain host memory pool instead of HostPoolGroup — i.e. dynamic sidecar pools were not enabled at construction time.

Common situations: Programmatically registering sidecar host pools without enabling the HostPoolGroup configuration in server args / controller init; version changes where the default host pool type switched; custom setups constructing HybridCacheController manually with a single HiKVPool.

Related errors

• prefetch_threshold must be int, got {type(prefetch_threshold
• prefetch_timeout_base must be number, got {type(prefetch_tim
• prefetch_timeout_per_ki_token must be number, got {type(pref
• prefetch_timeout_max must be number, got {type(prefetch_time
• hicache_storage_pass_prefix_keys must be bool, got {type(hic

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ce62a6ee51eec1c0. Report an issue: GitHub.