What causes "Host reference counter is already zero." in sgl-project/sglang?

Audit call sites to guarantee each lock_host() has exactly one matching release_host()

sgl-project/sglang · error · RuntimeError

Host reference counter is already zero.

Error message

Host reference counter is already zero.

What it means

TreeNode.release_host() raises RuntimeError when the host (CPU) reference counter is already zero, i.e. the node's host value is not currently pinned. This guards against double-release of host-side KV cache values in the hierarchical cache.

Source

Thrown at python/sglang/srt/mem_cache/radix_cache.py:285

    @property
    def evicted(self):
        return self.value is None

    @property
    def backuped(self):
        return self.host_value is not None

    def protect_host(self):
        """Protect the host value from eviction."""
        self.host_ref_counter += 1

    def release_host(self):
        """Release the host value, allowing it to be evicted."""
        if self.host_ref_counter > 0:
            self.host_ref_counter -= 1
        else:
            raise RuntimeError("Host reference counter is already zero.")

    def get_last_hash_value(self) -> Optional[str]:
        """Returns the hash value of the last page in this node."""
        if self.hash_value is None or len(self.hash_value) == 0:
            return None
        return self.hash_value[-1]

    def get_prefix_hash_values(self, node: TreeNode) -> List[str]:
        if node is None or node.hash_value is None:
            return []

        return node.get_prefix_hash_values(node.parent) + node.hash_value

    def __lt__(self, other: TreeNode):
        return self.last_access_time < other.last_access_time


class RadixCache(BasePrefixCache):

View on GitHub (pinned to 0132848349)

Solutions

Audit call sites to guarantee each lock_host() has exactly one matching release_host()
Guard with: if node.host_ref_counter > 0: node.release_host()
Check for duplicate cleanup in retraction + eviction handlers and deduplicate

Example fix

# before
node.release_host()
# after
if node.host_ref_counter > 0:
    node.release_host()

Defensive patterns

Strategy: type-guard

Validate before calling

if node.host_ref_counter > 0:
    node.release_host()

Type guard

def can_release_host(node) -> bool:
    return node.host_ref_counter > 0

Try / catch

try:
    node.release_host()
except RuntimeError:
    logger.warning("host ref already zero for node %s", node)
    # idempotent cleanup: ignore

Prevention

Pair every lock_host with exactly one release_host in a try/finally
Audit retraction paths for double release

When it happens

Trigger: Calling node.release_host() twice without an intervening lock_host(); releasing a node whose host value was already evicted or never written through to host.

Common situations: Retraction/eviction paths that race with request completion and both call release_host; refactored code paths that release host refs in a loop plus an unconditional cleanup; bug in ref-count pairing after a cache_finished_req.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/147503eaa8bf2e88. Report an issue: GitHub.