sgl-project/sglang · error · ValueError

invalid Kimi-K3 attention-residual target {target_index}

Error message

invalid Kimi-K3 attention-residual target {target_index}

What it means

Raised by _residual_target_value in Kimi-K3 GGUF weight conversion when the attention-residual target index is neither 0 nor 1. The target index determines whether the raw vector or a ones-like tensor is emitted for the dual residual parameters, so any other value is a programming error in the target mapping.

Source

Thrown at python/sglang/srt/model_loader/kimi_k3_gguf.py:126

    return (prefix + target,)


def _runtime_name(checkpoint_name: str, quantized: bool) -> str:
    if not quantized or not checkpoint_name.endswith(".weight"):
        return checkpoint_name
    return checkpoint_name.removesuffix("weight") + "qweight"


def _residual_target_value(raw: torch.Tensor, target_index: int) -> torch.Tensor:
    if raw.ndim != 1:
        raise ValueError(
            f"Kimi-K3 attention-residual score must be a vector, got {tuple(raw.shape)}"
        )
    if target_index == 0:
        return raw.unsqueeze(0)
    if target_index == 1:
        return torch.ones_like(raw)
    raise ValueError(f"invalid Kimi-K3 attention-residual target {target_index}")


def _kda_a_log_target_value(raw: torch.Tensor) -> torch.Tensor:
    """Undo llama.cpp's GGUF-time ``A_log -> -exp(A_log)`` transform."""
    if not raw.is_floating_point() or not torch.isfinite(raw).all():
        raise ValueError("Kimi-K3 GGUF ssm_a must contain finite floating values")
    if not torch.all(raw < 0):
        raise ValueError("Kimi-K3 GGUF ssm_a must contain only -exp(A_log) values")
    return torch.log(-raw)


def kimi_k3_nonexpert_weights_iterator(
    manifest_path: str | os.PathLike[str],
) -> Generator[tuple[str, torch.Tensor], None, None]:
    """Stream non-routed tensors shard by shard without reading routed payloads."""

    import gguf

View on GitHub (pinned to 0132848349)

Solutions

Check kimi_k3_checkpoint_targets: any tensor resolving to exactly 2 targets must keep target index 0 or 1; remove extra targets.
If you added a new residual parameter, extend _residual_target_value with an explicit branch for the new index instead of relying on the ValueError.
Run the unit tests test_residual_score_preserves_exact_combined_weight / test_residual_score_rejects_non_vector_source to verify the mapping.

Example fix

// before
targets = kimi_k3_checkpoint_targets(name)  # returns 3 targets
...
value = _residual_target_value(raw, target_index)  # target_index == 2 -> raises

// after
targets = kimi_k3_checkpoint_targets(name)  # keep exactly [score, ones] targets
assert target_index in (0, 1)
value = _residual_target_value(raw, target_index)

Defensive patterns

Strategy: validation

Validate before calling

targets = kimi_k3_checkpoint_targets(name)
if len(targets) == 2:
    assert all(i in (0, 1) for i in range(len(targets))), f"bad targets for {name}"

Type guard

def is_valid_residual_index(i: int) -> bool:
    return i in (0, 1)

Prevention

Keep residual target lists exactly length 2 and covered by unit tests.
Never reorder target names without re-running the residual tests.

When it happens

Trigger: Calling _residual_target_value(raw, target_index) with target_index not in {0,1}; indirectly via kimi_k3_nonexpert_weights_iterator when kimi_k3_checkpoint_targets maps a checkpoint tensor to exactly 2 targets but the iteration index goes out of range.

Common situations: Editing the residual target mapping (kimi_k3_checkpoint_targets) and adding a third target name, or reordering targets so the enumerate index no longer matches 0/1; bugs in custom fork of the loader.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/492195c787de316d. Report an issue: GitHub.