sgl-project/sglang · error · ValueError

Host-pool retraction does not support Mamba models.

Error message

Host-pool retraction does not support Mamba models.

What it means

When disagg decode retraction backup is set to 'host_pool', the unified radix cache factory rejects hybrid SSM (Mamba/Mamba2-style) models: their linear-attention state cannot be backed up to the host pool the way KV pages can. It raises ValueError at cache construction time.

Source

Thrown at python/sglang/srt/mem_cache/registry.py:154

        # Honor a CLI --flexkv-config-file by forwarding it via the env
        # var that FlexKV's config loader actually reads.
        if get_memory().flexkv_config_file and not os.environ.get("FLEXKV_CONFIG_PATH"):
            os.environ["FLEXKV_CONFIG_PATH"] = get_memory().flexkv_config_file
        return _flexkv_factory(ctx)

    return _create_unified_radix_cache(ctx, server_args, params)


def _create_unified_radix_cache(
    ctx: TreeCacheBuildContext,
    server_args: ServerArgs,
    params: CacheInitParams,
) -> BasePrefixCache:
    """Initialize a UnifiedRadixCache with proper components and optional HiCache."""
    if get_disagg().disaggregation_decode_retraction_backup == "host_pool":
        if ctx.is_hybrid_ssm:
            raise ValueError("Host-pool retraction does not support Mamba models.")
        if ctx.is_hybrid_swa and ctx.full_tokens_per_layer == 0:
            raise ValueError("Host-pool retraction does not support pure-SWA models.")

    from sglang.srt.mem_cache.unified_cache.components import ComponentType
    from sglang.srt.mem_cache.unified_radix_cache import UnifiedRadixCache

    tree_components = [ComponentType.FULL]
    if ctx.is_hybrid_swa:
        tree_components.append(ComponentType.SWA)
    if ctx.is_hybrid_ssm:
        tree_components.append(ComponentType.MAMBA)

    if hasattr(params.req_to_token_pool, "req_to_c128_sidecar"):
        from sglang.srt.hardware_backend.npu.dsv4.c128_sidecar_component import (
            C128SidecarComponent,
        )

        tree_components.append(ComponentType.C128)

View on GitHub (pinned to 0132848349)

Solutions

1. Use a different retraction backup mode (e.g. default) for hybrid SSM models
2. Per-model config: disable host_pool backup for Mamba hybrids and keep it only for attention models
3. Upgrade if a later version adds SSM state backup support

Example fix

# before
SGLANG_DISAGG_DECODE_RETRACTION_BACKUP=host_pool  # with Qwen3-Next / Mamba hybrid
# after
# unset or set to default for hybrid SSM models

if get_disagg().disaggregation_decode_retraction_backup == "host_pool" and model_is_hybrid_ssm:
    # fall back to default retraction
    set_retraction_backup("default")

When it happens

Trigger: Setting disaggregation_decode_retraction_backup='host_pool' (env/disagg config) and loading a hybrid SSM model (e.g. Qwen3-Next, Falcon-H1, a Mamba hybrid) so ctx.is_hybrid_ssm is True.

Common situations: PD-disaggregation deployments tuning retraction behavior with host-pool backup enabled globally, then scheduling a Mamba hybrid model on the same cluster/config.

Related errors

• Host-pool retraction does not support pure-SWA models.
• PD state transfer failed: kv_args.state_types is empty but s
• PD state transfer failed: mamba requires single state index,
• --hicache-host-memory-mode buffer_only is only implemented f
• --enable-session-radix-cache requires UnifiedRadixCache, but

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2f42a01d0ead73b1. Report an issue: GitHub.