sgl-project/sglang · error · ValueError

Unregistered UMBP hybrid pool: {}

Error message

Unregistered UMBP hybrid pool: {}

What it means

During hybrid (MLA/MHA) page key composition, the pool name on a PoolTransfer was not found in registered_pools, so per-page suffix keys cannot be built. Each transfer must reference a pool registered with the UMBP store.

Source

Thrown at python/sglang/srt/mem_cache/storage/umbp/umbp_store.py:1245

    # logical KV anchor that owns only page indices. The controller registers
    # each real pool through register_mem_host_pool_v2() and drives storage
    # via these _v2 methods, one PoolTransfer per pool. This mirrors the proven
    # MooncakeStore / HiCacheHF3FS design, specialized for UMBP's page_first,
    # single-object-per-page layout (each page -> exactly one storage object).
    # ------------------------------------------------------------------
    def _get_hybrid_page_component_keys(self, page_keys, transfer: PoolTransfer):
        """Map per-page logical keys to per-object storage keys for a side pool.

        For UMBP every registered side pool is page_first and stores one object
        per page (MLA: a single K object; MHA: a K and a V object), so the
        component-key count is an exact multiple of the page count. The pool
        name is embedded in the suffix so pages that share a hash across pools
        never collide.
        """
        pool_name = transfer.name
        host_pool = self.registered_pools.get(pool_name)
        if host_pool is None:
            raise ValueError(f"Unregistered UMBP hybrid pool: {pool_name}")

        if self.is_mla_backend:
            # Single compressed object per page.
            suffixes = [f"_{self.mla_suffix}_{pool_name}"]
        elif getattr(host_pool, "v_buffer", None) is not None:
            # Ordinary MHA side pool mirrors a K/V pool.
            suffixes = [
                f"_{self.mha_suffix}_{pool_name}_k",
                f"_{self.mha_suffix}_{pool_name}_v",
            ]
        else:
            suffixes = [f"_{self.mha_suffix}_{pool_name}"]

        key_multiplier = len(suffixes)
        component_keys = [
            f"{page_key}{suffix}" for page_key in page_keys for suffix in suffixes
        ]
        return component_keys, key_multiplier

View on GitHub (pinned to 0132848349)

Solutions

  1. Ensure every pool referenced by transfers is registered (registered_pools) before issuing batch I/O
  2. Check for name typos/mismatch between the pool registration and the PoolTransfer construction
  3. Register pools during model runner init before scheduler-driven I/O starts

Example fix

// before
transfer = PoolTransfer(name='kv_pool_mha', ...)
store.batch_get_v2([transfer])
// after
store.registered_pools['kv_pool_mha'] = host_pool  # ensure registered first
transfer = PoolTransfer(name='kv_pool_mha', ...)  # must match exactly
store.batch_get_v2([transfer])
Defensive patterns

Strategy: validation

Validate before calling

assert transfer.name in store.registered_pools, f"pool {transfer.name!r} not registered"

Type guard

def transfer_pool_registered(store, t) -> bool:
    return t.name in store.registered_pools

Prevention

When it happens

Trigger: Calling batch_exists_v2 or _batch_io_v2 with a PoolTransfer whose .name doesn't match any registered host pool (typo, unregistered pool, or registration skipped).

Common situations: Adding a new hybrid pool but forgetting to register it with the UMBP store; renaming pools between components; race where transfer is built before registration completes.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ce9397fa5e0e4c28. Report an issue: GitHub.