sgl-project/sglang · error · ValueError

BatchedDecodeContext requires full_kv_pool_index_by_layer wh

Error message

BatchedDecodeContext requires full_kv_pool_index_by_layer when the fused AOT RoPE + pool-scatter kernel is active

What it means

When the fused AOT RoPE + pool-scatter kernel is enabled (`self.aot.rope is not None`), the scatter writes into full-attention pool buffers, so the decode context must carry a `full_kv_pool_index_by_layer` mapping. Without it the kernel would fall back to per-cache indices and write to the wrong buffers when sliding-window layers are interleaved with full-attention layers. The dataclass's __post_init__ therefore validates this invariant.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/kv_cache/attention_wrapper.py:82

    def __post_init__(self) -> None:
        seq_lens = self.seq_lens
        max_seq_len = max(seq_lens)
        self.offsets = mx.array(seq_lens, dtype=mx.int32)
        self.max_len = max_seq_len + 1
        self.valid_lens = self.offsets + 1
        self.needs_padding = min(seq_lens) < max_seq_len
        self.pad_sizes = [max_seq_len - s for s in seq_lens]
        self.positions = mx.arange(self.max_len) if self.needs_padding else None
        if not self.attention_pool_index_by_layer:
            self.attention_pool_index_by_layer = {
                idx: idx for idx in range(len(self.attention_layer_caches))
            }
        if self.aot.rope is not None and not self.full_kv_pool_index_by_layer:
            # The fused scatter addresses pool buffers by full-attention index;
            # defaulting to the cache index would write the wrong buffer
            # whenever sliding-window layers are interleaved.
            raise ValueError(
                "BatchedDecodeContext requires full_kv_pool_index_by_layer "
                "when the fused AOT RoPE + pool-scatter kernel is active"
            )

    def decode_padding(
        self, window: int | None
    ) -> tuple[list[int], Optional[mx.array]]:
        """Right-pad sizes and the keep-mask for one decode step.

        Requests are padded to a common KV width so they can be batched into
        one SDPA call.  Without a window that width is ``max_len``; a
        sliding-window layer only reads the trailing ``window`` keys, so its
        width is ``max(min(seq_len + 1, window))`` instead -- which is why the
        context's full-length metadata cannot be reused for it.

        The mask is boolean (``True`` keeps the key), broadcast-shaped
        ``(B, 1, 1, width)``, and ``None`` when no request needs padding.
        """

View on GitHub (pinned to 0132848349)

Solutions

1. Populate `full_kv_pool_index_by_layer` (layer index -> full-attention KV pool index) whenever building the context with the fused AOT kernel active.
2. If the fused kernel isn't needed, disable it (unset SGLANG_MLX_USE_CUSTOM_ROPE) so `aot.rope` is None.
3. Update third-party/copy-pasted BatchedDecodeContext constructions to the new required field after upgrades.

Example fix

# before
ctx = BatchedDecodeContext(aot=aot_with_rope, ...)

# after
ctx = BatchedDecodeContext(
    aot=aot_with_rope,
    full_kv_pool_index_by_layer={i: pool.layout.full_pool_index(i) for i in attention_layers},
    ...,
)

if aot.rope is not None and not full_kv_pool_index_by_layer:
    full_kv_pool_index_by_layer = build_full_pool_index_map(layout)
ctx = BatchedDecodeContext(aot=aot, full_kv_pool_index_by_layer=full_kv_pool_index_by_layer, ...)

When it happens

Trigger: Constructing `BatchedDecodeContext` with `aot.rope` set but leaving `full_kv_pool_index_by_layer` empty/None — typically when a caller builds the context manually or a code path forgets to populate the layer→pool-index map for hybrid-window models.

Common situations: Enabling SGLANG_MLX_USE_CUSTOM_ROPE / the AOT fused kernel on a model with interleaved sliding-window layers; upgrading SGLang where BatchedDecodeContext gained a new required field; custom decode loops constructing the context directly.

Understand the failure class

Background: "Missing required field" and "field is required" errors: why libraries reject payloads that omit mandatory fields — this error's family across 20 libraries.

Related errors

• kv-canary: RealKvSource.page_size must be >= 1, got {self.pa
• realtime_causal_sink_size must be non-negative
• realtime_causal_kv_cache_num_frames must be positive
• sgl_kernel.metal is importable, but the native Metal extensi
• AttentionOffsetCache should not store data

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a95989c29921c09b. Report an issue: GitHub.