sgl-project/sglang · error · ValueError

Kimi-K3 GGUF ssm_a must contain only -exp(A_log) values

Error message

Kimi-K3 GGUF ssm_a must contain only -exp(A_log) values

What it means

Raised by _kda_a_log_target_value when the ssm_a tensor contains non-negative values. Since the loader inverts A_log -> -exp(A_log) (which is always strictly negative), any value >= 0 means the tensor was not transformed the way llama.cpp transforms it, and taking torch.log(-raw) would be invalid.

Source

Thrown at python/sglang/srt/model_loader/kimi_k3_gguf.py:134

def _residual_target_value(raw: torch.Tensor, target_index: int) -> torch.Tensor:
    if raw.ndim != 1:
        raise ValueError(
            f"Kimi-K3 attention-residual score must be a vector, got {tuple(raw.shape)}"
        )
    if target_index == 0:
        return raw.unsqueeze(0)
    if target_index == 1:
        return torch.ones_like(raw)
    raise ValueError(f"invalid Kimi-K3 attention-residual target {target_index}")


def _kda_a_log_target_value(raw: torch.Tensor) -> torch.Tensor:
    """Undo llama.cpp's GGUF-time ``A_log -> -exp(A_log)`` transform."""
    if not raw.is_floating_point() or not torch.isfinite(raw).all():
        raise ValueError("Kimi-K3 GGUF ssm_a must contain finite floating values")
    if not torch.all(raw < 0):
        raise ValueError("Kimi-K3 GGUF ssm_a must contain only -exp(A_log) values")
    return torch.log(-raw)


def kimi_k3_nonexpert_weights_iterator(
    manifest_path: str | os.PathLike[str],
) -> Generator[tuple[str, torch.Tensor], None, None]:
    """Stream non-routed tensors shard by shard without reading routed payloads."""

    import gguf

    manifest_file = Path(manifest_path).resolve()
    manifest = json.loads(manifest_file.read_text(encoding="utf-8"))
    if manifest.get("format") != "SGLANG-KIMI-GGMLMOEPACK-ADAPTER-v1":
        raise ValueError("Kimi-K3 manifest format is unsupported")
    if not manifest.get("complete"):
        raise ValueError("Kimi-K3 manifest is incomplete")

    records_by_shard: dict[int, list[dict]] = defaultdict(list)

View on GitHub (pinned to 0132848349)

Solutions

1. Regenerate the GGUF with a converter that applies A_log -> -exp(A_log) (llama.cpp-compatible export).
2. Inspect the tensor: if all values look like logs (mixed sign, small magnitude) it is raw A_log — re-export with the transform.
3. Check tensor naming in the manifest/records matches the converter's scheme.

Example fix

# before: writer.add_tensor("ssm_a", torch.log(a))           # wrong: raw A_log
# after:  writer.add_tensor("ssm_a", -torch.exp(a_log))        # llama.cpp transform

if not bool((raw < 0).all()):
    raise SystemExit("ssm_a not in -exp(A_log) form; re-export the GGUF")

def is_neg_exp_form(t: torch.Tensor) -> bool:
    return bool((t < 0).all())

When it happens

Trigger: The GGUF ssm_a tensor contains zero or positive entries — e.g. it was written from raw A_log values instead of -exp(A_log), or mixed-up tensor names map the wrong data into ssm_a.

Common situations: Using a GGUF produced by a converter that does not apply the llama.cpp A_log transform; tensor renamed/mis-mapped so another weight lands in the ssm_a slot; hand-edited GGUF.

Related errors

• invalid Kimi-K3 attention-residual target {target_index}
• Kimi-K3 GGUF ssm_a must contain finite floating values
• Kimi-K3 manifest format is unsupported
• Kimi-K3 manifest is incomplete
• attn_res: nvb must be in [1, {_MAX_BANK_ROWS}], got {nvb}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9435a92050adb818. Report an issue: GitHub.