sgl-project/sglang · error · ValueError

Unknown Shard Id {shard_id}

Error message

Unknown Shard Id {shard_id}

What it means

adjust_scalar_to_fused_array broadcasts a scalar quant scale into a fused QKV scale array; it maps shard_id 'q'/'k'/'v' (or 0/1/2) to the output slot. Any shard_id that is neither a string key in qkv_idxs nor an int raises ValueError('Unknown Shard Id ...').

Source

Thrown at python/sglang/srt/layers/linear.py:121

    quantized_offset = orig_offset * quantized_total // total
    quantized_size = orig_size * quantized_total // total

    return quantized_size, quantized_offset


def adjust_scalar_to_fused_array(param, loaded_weight, shard_id):
    """For fused modules (QKV and MLP) we have an array of length
    N that holds 1 scale for each "logical" matrix. So the param
    is an array of length N. The loaded_weight corresponds to
    one of the shards on disk. Here, we slice the param based on
    the shard_id for loading.
    """
    qkv_idxs = {"q": 0, "k": 1, "v": 2}

    if isinstance(shard_id, str):
        shard_id = qkv_idxs[shard_id]
    elif not isinstance(shard_id, int):
        raise ValueError(f"Unknown Shard Id {shard_id}")

    # AutoFP8 scales do not have a shape
    # compressed-tensors scales do have a shape
    if len(loaded_weight.shape) != 0:
        assert loaded_weight.shape[0] == 1
        loaded_weight = loaded_weight[0]

    return param[shard_id], loaded_weight


def adjust_shard_offsets(shard_offsets, loaded_weight, dim):
    actual_weight_size = loaded_weight.size(dim)
    target_weight_size = shard_offsets[-1][-1] + shard_offsets[-1][-2]
    if actual_weight_size != target_weight_size:
        new_shard_offsets = []
        new_offset = 0
        for shard_id, shard_offset, shard_size in shard_offsets:
            actual_shard_size = actual_weight_size * shard_size // target_weight_size

View on GitHub (pinned to 0132848349)

Solutions

  1. Pass shard_id as 'q', 'k', 'v' or int 0/1/2
  2. For non-QKV fused layers (gate/up), use a weight_loader path that handles those shard ids or weight_loader_v2
  3. If shard_id is a tuple, route to weight_loader_v2 as the code itself suggests

Example fix

# before
linear.weight_loader(param, scale, loaded_shard_id=("q", 0))

# after
linear.weight_loader(param, scale, loaded_shard_id="q")
Defensive patterns

Strategy: type-guard

Validate before calling

valid = {"q", "k", "v", 0, 1, 2}
assert loaded_shard_id in valid, f"bad shard_id {loaded_shard_id!r}; expected q/k/v or 0/1/2"
linear.weight_loader(param, loaded_weight, loaded_shard_id)

Type guard

def is_valid_qkv_shard_id(sid) -> bool:
    return sid in ("q", "k", "v") or (isinstance(sid, int) and not isinstance(sid, bool) and 0 <= sid <= 2)

Try / catch

try:
    linear.weight_loader(param, loaded_weight, loaded_shard_id=sid)
except ValueError as e:
    if "Unknown Shard Id" in str(e):
        sid = {"gate": 0, "up": 1}.get(sid, sid)  # map or fix upstream
        linear.weight_loader_v2(param, loaded_weight, loaded_shard_id=sid)
    else:
        raise

Prevention

When it happens

Trigger: Calling weight_loader for a fused QKV quant scale with loaded_shard_id of an unexpected type/value, e.g. a model implementation passing a tuple, an Enum, or a shard name like 'gate'/'up'/'o' instead of 'q'/'k'/'v' or 0/1/2.

Common situations: New model architectures reusing MergedColumnParallelLinear-style scale loading for non-QKV fused layers (gate/up projections); refactors changing shard_id to a tuple, which this v1 path does not support.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e8394b6c0ebc5f26. Report an issue: GitHub.