sgl-project/sglang · error · ValueError

Unsupported IO backend: {io_backend}

Error message

Unsupported IO backend: {io_backend}

What it means

load_to_device_per_layer received an io_backend value outside the supported set {'triton'/'kernel', 'direct', 'kernel_ascend'} for the MLA host pool, so no host-to-device transfer path can be selected.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/mla.py:339

            if self.layout == "page_first_kv_split":
                # Ascend-specific: transfer KV data for all layers when layer_id == 0
                if device_layer_id == 0:
                    transfer_kv_dim_exchange(
                        device_indices=device_indices,
                        host_indices=host_indices,
                        device_k=device_pool.k_buffer,
                        host_k=self.k_buffer,
                        device_v=device_pool.v_buffer,
                        host_v=self.v_buffer,
                        device_index_k=device_pool.index_k_buffer,
                        host_index_k=self.index_k_buffer,
                        page_size=self.page_size,
                        direction=TransferDirection.H2D,
                    )
            else:
                raise ValueError(f"Unsupported layout: {self.layout}")
        else:
            raise ValueError(f"Unsupported IO backend: {io_backend}")

    def _backup_from_device_per_layer(
        self,
        device_pool,
        host_indices,
        device_indices,
        layer_id,
        io_backend,
        *,
        is_draft: bool = False,
    ):
        # Indices arrive already translated by backup_from_device_all_layer.
        # MTP draft layers do not participate in CP layer sharding.
        host_layer_id = layer_id if is_draft else self._host_layer_index(layer_id)
        device_layer_id = 0 if is_draft else layer_id

        if io_backend == "kernel":
            if self.layout == "layer_first":

View on GitHub (pinned to 0132848349)

Solutions

1. Set io_backend to one of 'triton', 'direct', or 'kernel_ascend'
2. Trace where io_backend is passed from (server args / HiCache config) and fix the propagated value
3. Add an early validation of io_backend at startup

Example fix

// before
io_backend = "cuda"  # unsupported

// after
io_backend = "direct"

MLA_IO_BACKENDS = {"triton", "direct", "kernel_ascend"}
if io_backend not in MLA_IO_BACKENDS:
    raise ConfigError(f"io_backend must be one of {MLA_IO_BACKENDS}, got {io_backend!r}")

from typing import Literal
IoBackend = Literal["triton", "direct", "kernel_ascend"]

try:
    pool.load_to_device_per_layer(..., io_backend=io_backend)
except ValueError as e:
    if "Unsupported IO backend" in str(e):
        io_backend = "direct"
        pool.load_to_device_per_layer(..., io_backend=io_backend)
    else:
        raise

When it happens

Trigger: Calling load_to_device_per_layer(..., io_backend=X) with X not one of the implemented branches (typo, None, or a backend added elsewhere but not here).

Common situations: HiCache io_backend server arg typo or version skew between where the arg is validated and where it is consumed; experimental backends not wired into the MLA pool.

Related errors

• Unsupported layout for models with head_dim != v_head_dim an
• Unsupported layout for models with head_dim != v_head_dim an
• Unsupported IO backend for models with head_dim != v_head_di
• Unsupported V4 paged host layout/backend: {self.layout}/{io_
• Unsupported layout: {self.layout}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3f5d60d90dbf932a. Report an issue: GitHub.