sgl-project/sglang · error · RuntimeError

Staging is enabled but kv_manager._staging_ctx.allocator is

Error message

Staging is enabled but kv_manager._staging_ctx.allocator is None. Check that the transfer backend correctly initializes the staging allocator.

What it means

DecodeStagingHandler.create reads kv_manager._staging_ctx.allocator and refuses to construct the handler when the staging allocator is None. The staging-buffer feature (SGLANG_DISAGG_STAGING_BUFFER) requires the active transfer backend to have allocated the staging pool during kv_manager initialization.

Source

Thrown at python/sglang/srt/disaggregation/common/staging_handler.py:129

            self._wm_subscribers[key] = (receiver, session_id)

    def num_writers_for(self, receiver) -> int:
        """Compute all TP and PP writers expected for a staging chunk."""
        prefill_info = receiver.prefill_info
        prefill_tp = prefill_info.attn_tp_size
        if prefill_tp > self.decode_tp:
            tp_writers = prefill_tp // max(1, self.decode_tp)
        else:
            tp_writers = 1
        pp_writers = prefill_info.pp_size // self.kv_manager.pp_size
        return tp_writers * pp_writers

    @classmethod
    def create(cls, kv_manager, scheduler, tp_rank: int) -> DecodeStagingHandler:
        """Factory: create handler. Raises if staging infra is missing."""
        staging_allocator = kv_manager._staging_ctx.allocator
        if staging_allocator is None:
            raise RuntimeError(
                "Staging is enabled but kv_manager._staging_ctx.allocator is None. "
                "Check that the transfer backend correctly initializes the staging allocator."
            )
        kv_buffer_info = kv_manager.kv_buffer_tensors
        if kv_buffer_info is None:
            raise RuntimeError(
                "Staging is enabled but kv_manager.kv_buffer_tensors is None. "
                "Check that set_kv_buffer_tensors() was called during kv_manager init."
            )
        decode_tp = kv_manager.attn_tp_size

        from sglang.srt.disaggregation.common.staging_buffer import (
            resolve_total_kv_heads,
        )

        total_kv_heads = resolve_total_kv_heads(kv_manager.kv_args, decode_tp)
        return cls(
            kv_manager=kv_manager,

View on GitHub (pinned to 0132848349)

Solutions

1. Switch to a transfer backend that supports the staging allocator (check backend init code for _staging_ctx.allocator assignment)
2. Unset SGLANG_DISAGG_STAGING_BUFFER if you don't need staging
3. If developing a backend, assign kv_manager._staging_ctx.allocator during kv_manager init before transfer_queue._init_staging_handler runs

Example fix

# before
export SGLANG_DISAGG_STAGING_BUFFER=1  # backend without staging support
# after
unset SGLANG_DISAGG_STAGING_BUFFER

if envs.SGLANG_DISAGG_STAGING_BUFFER.get():
    assert kv_manager._staging_ctx.allocator is not None, 'backend lacks staging allocator'

When it happens

Trigger: Setting SGLANG_DISAGG_STAGING_BUFFER=1 on a decode server whose transfer backend does not (or not yet) initialize _staging_ctx.allocator — e.g. an unsupported backend or initialization-order bug.

Common situations: Enabling the staging flag with a backend that never implemented staging; a backend refactor that deferred/removed allocator setup; race where handler creation precedes backend init.

Related errors

• Staging is enabled but kv_manager.kv_buffer_tensors is None.
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models
• Cannot resolve total_kv_heads: kv_args has neither total_kv_
• [Staging] KV transfer via staging buffer failed: {e}. sessio
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/99348e515f77df70. Report an issue: GitHub.