sgl-project/sglang · error · ValueError

Host-pool retraction does not support pure-SWA models.

Error message

Host-pool retraction does not support pure-SWA models.

What it means

The unified radix cache factory rejects pure-SWA (sliding-window attention only) models when disagg decode retraction backup is 'host_pool': with full_tokens_per_layer == 0 there is no full-attention KV pool to back up to host, so host-pool retraction is meaningless. Only hybrid SWA models with a nonzero full-attention pool can use it.

Source

Thrown at python/sglang/srt/mem_cache/registry.py:156

        # var that FlexKV's config loader actually reads.
        if get_memory().flexkv_config_file and not os.environ.get("FLEXKV_CONFIG_PATH"):
            os.environ["FLEXKV_CONFIG_PATH"] = get_memory().flexkv_config_file
        return _flexkv_factory(ctx)

    return _create_unified_radix_cache(ctx, server_args, params)


def _create_unified_radix_cache(
    ctx: TreeCacheBuildContext,
    server_args: ServerArgs,
    params: CacheInitParams,
) -> BasePrefixCache:
    """Initialize a UnifiedRadixCache with proper components and optional HiCache."""
    if get_disagg().disaggregation_decode_retraction_backup == "host_pool":
        if ctx.is_hybrid_ssm:
            raise ValueError("Host-pool retraction does not support Mamba models.")
        if ctx.is_hybrid_swa and ctx.full_tokens_per_layer == 0:
            raise ValueError("Host-pool retraction does not support pure-SWA models.")

    from sglang.srt.mem_cache.unified_cache.components import ComponentType
    from sglang.srt.mem_cache.unified_radix_cache import UnifiedRadixCache

    tree_components = [ComponentType.FULL]
    if ctx.is_hybrid_swa:
        tree_components.append(ComponentType.SWA)
    if ctx.is_hybrid_ssm:
        tree_components.append(ComponentType.MAMBA)

    if hasattr(params.req_to_token_pool, "req_to_c128_sidecar"):
        from sglang.srt.hardware_backend.npu.dsv4.c128_sidecar_component import (
            C128SidecarComponent,
        )

        tree_components.append(ComponentType.C128)
        params.component_registry_override = {
            **(params.component_registry_override or {}),

View on GitHub (pinned to 0132848349)

Solutions

1. Disable host_pool retraction backup for pure-SWA models (use the default backup mode)
2. Give the model a nonzero full-attention pool (hybrid config) if host backup is required
3. Gate the disagg setting per-model in your serving config

Example fix

# before
retraction_backup=host_pool  # pure SWA model
# after
retraction_backup=default  # or unset for pure SWA models

if retraction_backup == "host_pool" and ctx.is_hybrid_swa and ctx.full_tokens_per_layer == 0:
    set_retraction_backup("default")

When it happens

Trigger: disaggregation_decode_retraction_backup='host_pool' with ctx.is_hybrid_swa True and ctx.full_tokens_per_layer == 0 (a sliding-window-only model like Gemma-2/3-style SWA or a pure SWA config).

Common situations: Enabling host-pool retraction backup cluster-wide in a PD setup, then serving a pure SWA model that lacks a full KV pool.

Related errors

• Host-pool retraction does not support Mamba models.
• PD state transfer does not support TP-mismatched non-MLA SWA
• DSV4 draft state transfer expects SWA-only NextN layers
• DSV4 target and draft pools must share SWA ring geometry: ta
• DSV4 target and draft pools must share the SWA index mapping

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cc51dc204e1aa633. Report an issue: GitHub.