{"record":{"id":"4df3f65fc9382e8c","repo":"sgl-project/sglang","slug":"kimi-k3-gguf-ssm-a-must-contain-finite-floating-va","errorCode":null,"errorMessage":"Kimi-K3 GGUF ssm_a must contain finite floating values","messagePattern":"Kimi-K3 GGUF ssm_a must contain finite floating values","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/model_loader/kimi_k3_gguf.py","lineNumber":132,"sourceCode":"    return checkpoint_name.removesuffix(\"weight\") + \"qweight\"\n\n\ndef _residual_target_value(raw: torch.Tensor, target_index: int) -> torch.Tensor:\n    if raw.ndim != 1:\n        raise ValueError(\n            f\"Kimi-K3 attention-residual score must be a vector, got {tuple(raw.shape)}\"\n        )\n    if target_index == 0:\n        return raw.unsqueeze(0)\n    if target_index == 1:\n        return torch.ones_like(raw)\n    raise ValueError(f\"invalid Kimi-K3 attention-residual target {target_index}\")\n\n\ndef _kda_a_log_target_value(raw: torch.Tensor) -> torch.Tensor:\n    \"\"\"Undo llama.cpp's GGUF-time ``A_log -> -exp(A_log)`` transform.\"\"\"\n    if not raw.is_floating_point() or not torch.isfinite(raw).all():\n        raise ValueError(\"Kimi-K3 GGUF ssm_a must contain finite floating values\")\n    if not torch.all(raw < 0):\n        raise ValueError(\"Kimi-K3 GGUF ssm_a must contain only -exp(A_log) values\")\n    return torch.log(-raw)\n\n\ndef kimi_k3_nonexpert_weights_iterator(\n    manifest_path: str | os.PathLike[str],\n) -> Generator[tuple[str, torch.Tensor], None, None]:\n    \"\"\"Stream non-routed tensors shard by shard without reading routed payloads.\"\"\"\n\n    import gguf\n\n    manifest_file = Path(manifest_path).resolve()\n    manifest = json.loads(manifest_file.read_text(encoding=\"utf-8\"))\n    if manifest.get(\"format\") != \"SGLANG-KIMI-GGMLMOEPACK-ADAPTER-v1\":\n        raise ValueError(\"Kimi-K3 manifest format is unsupported\")\n    if not manifest.get(\"complete\"):\n        raise ValueError(\"Kimi-K3 manifest is incomplete\")","sourceCodeStart":114,"sourceCodeEnd":150,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/model_loader/kimi_k3_gguf.py#L114-L150","documentation":"Raised by _kda_a_log_target_value when undoing llama.cpp's GGUF-time A_log -> -exp(A_log) transform on a tensor that is not floating point or contains non-finite values (NaN/Inf). The KDA/SSM A_log recovery via torch.log(-raw) requires finite floats, so malformed or integer-quantized ssm_a data is rejected.","triggerScenarios":"kimi_k3_nonexpert_weights_iterator yields an ssm_a tensor that was stored as an integer/quantized GGUF type, or contains NaN/±Inf, and it is routed through _kda_a_log_target_value.","commonSituations":"Converting a Kimi-K3 GGUF file where ssm_a was quantized (e.g. to Q8) by the GGUF writer; corrupted shard file producing NaNs; a partial/interrupted GGUF conversion.","solutions":["Re-export or dequantize the GGUF so ssm_a (A_log) is stored as F32/F16/BF16 with finite values.","Verify shard integrity with gguf-py or the manifest size check before loading.","If NaNs persist, the source GGUF is corrupt — regenerate it from the original checkpoint."],"exampleFix":"# before: ssm_a stored quantized -> raises\n# after: keep ssm_a unquantized when writing GGUF\nwriter.add_tensor(\"ssm_a\", a_log_float32.numpy())  # F32, finite","handlingStrategy":"validation","validationCode":"raw = torch.tensor(tensor.data)\nassert raw.is_floating_point() and torch.isfinite(raw).all(), \"ssm_a not finite floats\"","typeGuard":"def is_finite_float(t: torch.Tensor) -> bool:\n    return t.is_floating_point() and bool(torch.isfinite(t).all())","tryCatchPattern":null,"preventionTips":["Keep ssm_a unquantized (F32/F16/BF16) when writing GGUF.","Validate GGUF tensors for finiteness right after reading the shard."],"tags":["kimi-k3","gguf","kda","mamba","nan"],"backgroundTag":"tensor-data-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}