sgl-project/sglang · error · RuntimeError

Staging is enabled but kv_manager.kv_buffer_tensors is None.

Error message

Staging is enabled but kv_manager.kv_buffer_tensors is None. Check that set_kv_buffer_tensors() was called during kv_manager init.

What it means

DecodeStagingHandler.create checks kv_manager.kv_buffer_tensors after the allocator check and raises if it is None. Staging transfers read/write through the kv_buffer_tensors handles, so they must have been registered via set_kv_buffer_tensors() during kv_manager init.

Source

Thrown at python/sglang/srt/disaggregation/common/staging_handler.py:135

        if prefill_tp > self.decode_tp:
            tp_writers = prefill_tp // max(1, self.decode_tp)
        else:
            tp_writers = 1
        pp_writers = prefill_info.pp_size // self.kv_manager.pp_size
        return tp_writers * pp_writers

    @classmethod
    def create(cls, kv_manager, scheduler, tp_rank: int) -> DecodeStagingHandler:
        """Factory: create handler. Raises if staging infra is missing."""
        staging_allocator = kv_manager._staging_ctx.allocator
        if staging_allocator is None:
            raise RuntimeError(
                "Staging is enabled but kv_manager._staging_ctx.allocator is None. "
                "Check that the transfer backend correctly initializes the staging allocator."
            )
        kv_buffer_info = kv_manager.kv_buffer_tensors
        if kv_buffer_info is None:
            raise RuntimeError(
                "Staging is enabled but kv_manager.kv_buffer_tensors is None. "
                "Check that set_kv_buffer_tensors() was called during kv_manager init."
            )
        decode_tp = kv_manager.attn_tp_size

        from sglang.srt.disaggregation.common.staging_buffer import (
            resolve_total_kv_heads,
        )

        total_kv_heads = resolve_total_kv_heads(kv_manager.kv_args, decode_tp)
        return cls(
            kv_manager=kv_manager,
            staging_allocator=staging_allocator,
            kv_buffer_info=kv_buffer_info,
            decode_tp=decode_tp,
            total_kv_heads=total_kv_heads,
            tp_rank=tp_rank,
            scheduler=scheduler,

View on GitHub (pinned to 0132848349)

Solutions

1. Call set_kv_buffer_tensors(...) on the kv_manager during its init (as the supported backends do)
2. Upgrade/patch the transfer backend so its kv_manager init registers kv_buffer_tensors
3. Disable SGLANG_DISAGG_STAGING_BUFFER if the feature is not needed

Example fix

# before
# kv_manager init ends without buffer registration
# after
kv_manager.set_kv_buffer_tensors(kv_buffer_tensors)

if envs.SGLANG_DISAGG_STAGING_BUFFER.get():
    assert kv_manager.kv_buffer_tensors is not None, 'set_kv_buffer_tensors not called'

When it happens

Trigger: Staging enabled (SGLANG_DISAGG_STAGING_BUFFER=1), allocator present, but set_kv_buffer_tensors() was never called on the kv_manager — a backend or init path that skips buffer-tensor registration.

Common situations: Backend integration that allocates the staging allocator but forgets to publish kv_buffer_tensors; init order change where the handler is created before buffer registration.

Understand the failure class

Background: "Missing required field" and "field is required" errors: why libraries reject payloads that omit mandatory fields — this error's family across 20 libraries.

Related errors

• Staging is enabled but kv_manager._staging_ctx.allocator is
• Cannot resolve total_kv_heads: kv_args has neither total_kv_
• [Staging] KV transfer via staging buffer failed: {e}. sessio
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models
• SGLANG_DISAGG_STAGING_BUFFER does not support prefill contex

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/18a1608d0b5dc90d. Report an issue: GitHub.