sgl-project/sglang · error · NotImplementedError

page-major layout has no per-layer contiguous regions; KV tr

Error message

page-major layout has no per-layer contiguous regions; KV transfer / disaggregation is unsupported (TODO: expose the single _raw buffer with a page-aware transfer scheme).

What it means

The page-major (4-D strided envelope) KV pool layout stores KV bytes interleaved layer-major within each page, so there is no per-layer contiguous 3-D region. Methods like get_contiguous_buf_infos assume that contiguous layout for KV transfer/disaggregation, so they raise NotImplementedError instead of silently mis-indexing.

Source

Thrown at python/sglang/srt/mem_cache/memory_pool.py:3313

    def _move_kv_cache_impl(self, tgt_loc: torch.Tensor, src_loc: torch.Tensor):
        # Strided 4-D views: the tiled copy kernel assumes stride == row bytes, so
        # always take the native move (it splits token ids into
        # (page_id, slot_in_page) for the 4-D advanced index).
        move_kv_cache_native(
            self.k_buffer,
            self.v_buffer,
            tgt_loc,
            src_loc,
            page_size=self.page_size,
        )

    # The methods below assume the per-layer contiguous 3-D layout. The 4-D
    # strided envelope views have no per-layer contiguous region (their bytes are
    # interleaved layer-major within each page) and index page-major, not
    # token-major. Inheriting them would silently mis-index; fail loudly instead.

    def get_contiguous_buf_infos(self):
        raise NotImplementedError(
            "page-major layout has no per-layer contiguous regions; KV transfer / "
            "disaggregation is unsupported (TODO: expose the single _raw buffer "
            "with a page-aware transfer scheme)."
        )

    def get_cpu_copy(self, indices, mamba_indices=None):
        raise NotImplementedError(
            "CPU offloading is unsupported under the page-major layout "
            "(TODO: split token ids into page/slot for the 4-D index)."
        )

    def load_cpu_copy(self, kv_cache_cpu, indices, mamba_indices=None):
        raise NotImplementedError(
            "CPU offloading is unsupported under the page-major layout "
            "(TODO: split token ids into page/slot for the 4-D index)."
        )

    def set_kv_buffer_prefix_valid(self, *args, **kwargs):

View on GitHub (pinned to 0132848349)

Solutions

Disable the page-major layout (fall back to page_size=1 / token-major 3-D pool) when disaggregation or KV transfer is needed
Disable disaggregation / KV transfer features if the page-major layout is required
Wait for / implement the page-aware transfer scheme over the single _raw buffer mentioned in the TODO

Defensive patterns

Strategy: validation

Validate before calling

def can_disaggregate(pool) -> bool:
    try:
        pool.get_contiguous_buf_infos()
        return True
    except NotImplementedError:
        return False

Type guard

def is_page_major_pool(pool) -> bool:
    return hasattr(pool, '_raw') and getattr(pool, 'page_size', 1) > 1

Try / catch

try:
    infos = pool.get_contiguous_buf_infos()
except NotImplementedError as e:
    logger.warning('KV transfer unsupported for %s: %s', type(pool).__name__, e)
    raise

Prevention

Add a startup capability check when disaggregation args are set
Document feature-compatibility matrix for page-major layout
Gate disaggregation flags on pool type in server_args validation

When it happens

Trigger: Enabling the page-major KV layout (page size > 1 with the strided 4-D buffer allocation) and then invoking get_contiguous_buf_infos, e.g. by starting a PD-disaggregation or KV transfer (cache transfer) flow on top of that pool.

Common situations: Turning on page-size/paged KV layout together with --disaggregation or any prefill-decode transfer feature; new code paths calling get_contiguous_buf_infos for transfer initialization.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/73fbed350272644f. Report an issue: GitHub.