sgl-project/sglang · critical · ValueError

expert-pack header coverage is inconsistent

Error message

expert-pack header coverage is inconsistent

What it means

The expert-pack binary header declares an index_count that does not equal num_layers * num_experts * 3 (ROLE_NAMES = gate/up/down), or role_count != 3. The reader (SGLANG-EXPERTPACK-v1) enforces that every (layer, expert, role) triplet has exactly one index entry, so any mismatch means the pack was built for a different topology or is corrupt.

Source

Thrown at python/sglang/srt/layers/moe/expert_pack.py:92

            raise ValueError("expert-pack magic or version does not match")
        if values[2] != HEADER_STRUCT.size or values[3] != ENTRY_STRUCT.size:
            raise ValueError("expert-pack struct sizes do not match")
        header = cls(
            flags=values[4],
            index_count=values[5],
            data_start=values[6],
            alignment=values[7],
            num_layers=values[8],
            num_experts=values[9],
            top_k=values[10],
            role_count=values[11],
            model_identity_sha256=values[12].hex(),
            source_blob_sha256=values[13].hex(),
            config_sha256=values[14].hex(),
        )
        expected = header.num_layers * header.num_experts * len(ROLE_NAMES)
        if header.index_count != expected or header.role_count != len(ROLE_NAMES):
            raise ValueError("expert-pack header coverage is inconsistent")
        if header.flags & REQUIRED_FLAGS != REQUIRED_FLAGS:
            raise ValueError("expert-pack is not identity triplet layout")
        if header.alignment <= 0 or header.alignment & (header.alignment - 1):
            raise ValueError("expert-pack alignment is invalid")
        minimum = HEADER_STRUCT.size + header.index_count * ENTRY_STRUCT.size
        if header.data_start < minimum or header.data_start % header.alignment:
            raise ValueError("expert-pack data offset is invalid")
        return header


@dataclass(frozen=True)
class ExpertPackEntry:
    layer: int
    expert: int
    role_id: int
    dtype_id: int
    dtype: str
    tensor_name: str

View on GitHub (pinned to 0132848349)

Solutions

  1. Regenerate the expert-pack so it matches the model's num_layers/num_experts and packs all three roles (gate, up, down)
  2. Verify the pack was produced by the matching version of the packing tool and fully transferred (compare sha256 with the manifest)
  3. If num_experts or num_layers of your model changed (e.g. different quant/config), rebuild the pack from the new checkpoint

Example fix

# before
store = ExpertPackStore("mymodel.expertpack", manifest_path=None)  # built for 8-expert model

# after
store = ExpertPackStore("mymodel.k8.expertpack")  # pack regenerated with matching num_layers/num_experts and role_count=3
Defensive patterns

Strategy: validation

Validate before calling

import struct
from sglang.srt.layers.moe.expert_pack import HEADER_STRUCT, ROLE_NAMES, MAGIC

def header_covers_topology(path, num_layers, num_experts):
    with open(path, "rb") as f:
        raw = f.read(HEADER_STRUCT.size)
    v = HEADER_STRUCT.unpack(raw)
    idx_count, n_layers, n_experts, role_count = v[5], v[8], v[9], v[11]
    return (idx_count == n_layers * n_experts * len(ROLE_NAMES)
            and role_count == len(ROLE_NAMES)
            and n_layers == num_layers and n_experts == num_experts)

Prevention

When it happens

Trigger: ExpertPackHeader.read(stream) on a pack whose header fields num_layers/num_experts/index_count/role_count disagree — e.g. a pack exported for a different model (num_experts 256 vs 8), a hand-edited header, or a truncated/rewritten file where role_count is not 3.

Common situations: Using an expert-pack built for a different MoE model or model revision; mixing packs from older/newer versions of the packer tool; partially transferred or corrupted pack files.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bda2344025517cf1. Report an issue: GitHub.