sgl-project/sglang · error · ValueError

Cannot resolve total_kv_heads: kv_args has neither total_kv_

Error message

Cannot resolve total_kv_heads: kv_args has neither total_kv_head_num nor kv_head_num. Ensure DecodePreallocQueue._init_kv_manager sets kv_args.kv_head_num.

What it means

resolve_total_kv_heads derives the total number of KV heads from kv_args by first reading total_kv_head_num, then falling back to kv_head_num * attn_tp_size. If both attributes are missing/zero, there is not enough metadata to compute per-layer transfer shapes, so it raises with a pointer to DecodePreallocQueue._init_kv_manager.

Source

Thrown at python/sglang/srt/disaggregation/common/staging_buffer.py:766

            dst_tp_rank,
            total_kv_heads,
        )
        writer_bytes.append(num_tokens * nh * bytes_per_head_token * num_layers * 2)
    return num_writers, writer_bytes, sum(writer_bytes)


def resolve_total_kv_heads(
    kv_args,
    attn_tp_size: int,
) -> int:
    """Resolve the global total KV head count from kv_args metadata."""
    total = getattr(kv_args, "total_kv_head_num", 0)
    if total > 0:
        return total
    per_rank = getattr(kv_args, "kv_head_num", 0)
    if per_rank > 0:
        return per_rank * attn_tp_size
    raise ValueError(
        "Cannot resolve total_kv_heads: kv_args has neither total_kv_head_num "
        "nor kv_head_num. "
        "Ensure DecodePreallocQueue._init_kv_manager sets kv_args.kv_head_num."
    )


def staging_grid_tokens(chunked_prefill_size: Optional[int], page_size: int) -> int:
    """Token width of one staging grid slot; shared by prefetch and the
    sender's grid alignment."""
    cps = chunked_prefill_size or 8192
    return max(1, cps // page_size) * page_size


def compute_grid_segments(
    start_idx: int, end_idx: int, base: int, grid_tokens: int
) -> List[Tuple[int, int]]:
    """Split [start_idx, end_idx) at grid boundaries base + k * grid_tokens
    so each segment maps to exactly one staging slot. An empty range yields

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure DecodePreallocQueue._init_kv_manager sets kv_args.kv_head_num (per-rank) or kv_args.total_kv_head_num before staging is used
2. If you build kv_args manually, populate both fields and pass the correct attn_tp_size
3. Add a startup assertion after kv_manager init to fail fast with clearer context

Example fix

# before
kv_args = KVArgs()  # kv_head_num never set
# after
kv_args.kv_head_num = total_kv_heads // attn_tp_size
kv_args.total_kv_head_num = total_kv_heads

assert getattr(kv_args, 'total_kv_head_num', 0) > 0 or getattr(kv_args, 'kv_head_num', 0) > 0, 'kv_args missing kv head counts'

When it happens

Trigger: Calling staging-buffer helpers (create, handle_staging_req, send_kvcache_staged) with a kv_args object lacking both total_kv_head_num and kv_head_num — typically when _init_kv_manager did not populate kv_head_num for a new backend or code path.

Common situations: New transfer backend integration that skips setting kv_args.kv_head_num; code paths that construct KVArgs fresh without the prealloc init; refactors dropping the field assignment.

Understand the failure class

Background: "Missing required field" and "field is required" errors: why libraries reject payloads that omit mandatory fields — this error's family across 20 libraries.

Related errors

• Staging is enabled but kv_manager._staging_ctx.allocator is
• Staging is enabled but kv_manager.kv_buffer_tensors is None.
• [Staging] KV transfer via staging buffer failed: {e}. sessio
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models
• SGLANG_DISAGG_STAGING_BUFFER does not support prefill contex

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9b0d4dd273bb44e5. Report an issue: GitHub.