sgl-project/sglang · error · RuntimeError
Missing aux index for last chunk
Error message
Missing aux index for last chunk
What it means
On the last chunk of a transfer, transfer_worker requires kv_chunk.prefill_aux_index to send the aux tensor; if it is None it raises RuntimeError. The aux index is populated during prefill request processing, so a None value on the final chunk means the chunk descriptor was built incompletely — an internal invariant break rather than an expected condition.
Source
Thrown at python/sglang/srt/disaggregation/nixl/conn.py:1284
state_xfer_handles = self.maybe_send_extra(
req.agent_name,
kv_chunk.state_indices,
dst_info.dst_state_data_ptrs,
req.dst_state_indices,
dst_info.gpu_id,
f"{req.room}_state_{self.transfer_source_rank}",
decode_tp_size,
decode_tp_rank=dst_info.decode_tp_rank,
dst_state_item_lens=dst_info.dst_state_item_lens,
dst_state_dim_per_tensor=dst_info.dst_state_dim_per_tensor,
dst_state_layer_ids=dst_info.dst_state_layer_ids,
)
handles.extend(
h for h in state_xfer_handles if h is not None
)
if kv_chunk.prefill_aux_index is None:
raise RuntimeError("Missing aux index for last chunk")
# A no-KV notification still identifies its PP source.
# Empty non-final chunks do not consume chunk IDs, so a
# final no-KV chunk_id equals the prior KV chunk count.
aux_notif = f"{req.room}_aux"
if (
len(kv_chunk.prefill_kv_indices) == 0
or not self.kv_args.kv_data_ptrs
):
aux_notif += (
f"_nokv_{self.transfer_source_rank}"
f"_{kv_chunk.chunk_id}"
)
aux_xfer_handle = self.send_aux(
req.agent_name,
kv_chunk.prefill_aux_index,
dst_info.dst_aux_ptrs,
req.dst_aux_index,
aux_notif,View on GitHub (pinned to 0132848349)
Solutions
- Verify the model/backend supports the aux tensors the NIXL path expects (check kv_args.aux_data_ptrs is populated)
- Check for and upgrade past SGLang versions with chunked-transfer aux-index bugs
- Reproduce with a minimal PD setup and report with the room/chunk logs
Defensive patterns
Strategy: try-catch
Validate before calling
if chunk.is_last_chunk:
assert chunk.prefill_aux_index is not None, 'chunker bug: aux index missing on final chunk' Try / catch
except RuntimeError as e:
if 'Missing aux index' in str(e):
mark_request_failed_and_retry_prefill(room) Prevention
- Unit-test chunk producers so the last chunk always carries aux metadata
- Keep chunk-producing and chunk-consuming components at the same version
- Watch for this on custom backends that skip aux buffer allocation
When it happens
Trigger: kv_chunk.is_last_chunk is True while kv_chunk.prefill_aux_index is None — happens if the chunk producer (prefill scheduler / KV chunker) failed to attach the aux index, or a custom/disaggregation bootstrap path constructs chunks without aux metadata.
Common situations: Bugs in chunked prefill or aux-tensor changes (e.g. a model/backend that does not allocate aux buffers, or a modified chunker); version mismatch between components that produce and consume chunk descriptors.
Related errors
- Missing NIXL destination KV memory kind
- Unexpected compressed-MLA dst_kv_ptrs length {len(dst_kv_ptr
- NIXL PD transfer does not support HiSparse combined with dec
- NIXL KV transfer has no KV memory segments
- NIXL heterogeneous-TP direct-to-host KV transfer is not impl
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/4af3f8d63788755d.
Report an issue: GitHub.