sgl-project/sglang · error · RuntimeError

SGLANG_DISAGG_STAGING_BUFFER requires a positive chunked_pre

Error message

SGLANG_DISAGG_STAGING_BUFFER requires a positive chunked_prefill_size that is a multiple of page_size ({page_size}); got {chunked_prefill_size}.

What it means

Startup validation for SGLANG_DISAGG_STAGING_BUFFER: the staging mechanism slices each KV send into a fixed page-aligned grid, so chunked_prefill_size must be positive and an exact multiple of the KV pool page_size. -1 (unbounded) or misaligned values have no valid grid and are rejected.

Source

Thrown at python/sglang/srt/disaggregation/prefill.py:173

        self.max_total_num_tokens = (
            self.scheduler.tp_worker.model_runner.effective_max_total_num_tokens
        )
        self.transfer_backend = transfer_backend
        if envs.SGLANG_DISAGG_STAGING_BUFFER.get():
            if self.is_mla_backend:
                raise RuntimeError(
                    "SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models "
                    "(e.g. GQA, MHA). MLA models should not set this flag."
                )
            page_size = self.scheduler.token_to_kv_pool_allocator.page_size
            # Same source as send_kv_chunk's staging grid below, so validation
            # and the grid cannot disagree after a post-publish override.
            chunked_prefill_size = get_schedule().chunked_prefill_size
            cps = chunked_prefill_size or 8192
            # Staging slices each send into a fixed page-aligned grid, so an
            # unbounded (-1) or non-page-aligned chunk size has no valid grid.
            if cps <= 0 or cps % page_size != 0:
                raise RuntimeError(
                    f"SGLANG_DISAGG_STAGING_BUFFER requires a positive "
                    f"chunked_prefill_size that is a multiple of page_size "
                    f"({page_size}); got {chunked_prefill_size}."
                )
            if self.pp_size > 1 and self.transfer_backend != TransferBackend.MOONCAKE:
                raise RuntimeError(
                    "SGLANG_DISAGG_STAGING_BUFFER with pp_size > 1 is only "
                    "supported by Mooncake."
                )
            if get_parallel().enable_prefill_context_parallel:
                # CP rewrites index_slice per rank, breaking the chunk grid.
                raise RuntimeError(
                    "SGLANG_DISAGG_STAGING_BUFFER does not support "
                    "prefill context parallelism."
                )
        self.kv_manager = self._init_kv_manager()

    def _init_kv_manager(self) -> CommonKVManager:

View on GitHub (pinned to 0132848349)

Solutions

Set --chunked-prefill-size to a positive multiple of page_size, e.g. if page_size=64 use 8192 or 16384.
Check the server log line for the page_size value and align chunked_prefill_size accordingly (or pass --page-size to make alignment easy).
Alternatively unset SGLANG_DISAGG_STAGING_BUFFER to disable staging and keep the current chunk size.

Example fix

# before
export SGLANG_DISAGG_STAGING_BUFFER=1
python -m sglang.launch_server --model qwen --chunked-prefill-size -1 ...
# after
export SGLANG_DISAGG_STAGING_BUFFER=1
python -m sglang.launch_server --model qwen --chunked-prefill-size 8192 --page-size 64 ...

Defensive patterns

Strategy: validation

Validate before calling

cps = chunked_prefill_size or 8192
assert cps > 0 and cps % page_size == 0, (
    f'chunked_prefill_size={cps} must be a positive multiple of page_size={page_size}')

Prevention

Derive chunked_prefill_size programmatically as page_size * N.
Never pass -1 chunked-prefill with staging buffer enabled.
Print page_size at launch and assert alignment in deploy scripts.

When it happens

Trigger: Setting SGLANG_DISAGG_STAGING_BUFFER=1 with --chunked-prefill-size -1, 0, or a value not divisible by the allocator page_size (default fallback 8192 is used when unset only if it divides evenly; any non-multiple trips the check).

Common situations: Default chunked_prefill_size (e.g. 8192, 16384) is not a multiple of page_size (e.g. 64) after a model/config change, or an operator explicitly passes -1 for unbounded prefill chunks while enabling the staging buffer.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9e5ee3a0c2c7ae6f. Report an issue: GitHub.