{"record":{"id":"df796c9593b13377","repo":"sgl-project/sglang","slug":"comfy-layer-prefix-r-is-missing-checkpoint-tenso","errorCode":null,"errorMessage":"Comfy layer {prefix!r} is missing checkpoint tensors: {sorted(missing)}","messagePattern":"Comfy layer (.+?) is missing checkpoint tensors: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":158,"sourceCode":"        if marker_format == \"asym_w4a8_int8\":\n            required = {\n                f\"{prefix}.weight\",\n                f\"{prefix}.weight_s_rel\",\n                f\"{prefix}.weight_s_channel\",\n            }\n        if marker_format == \"nvfp4\":\n            required.add(f\"{prefix}.weight_scale_2\")\n        if marker_format not in (\n            \"float8_e4m3fn\",\n            \"int8_tensorwise\",\n            \"asym_w4a8_int8\",\n            \"convrot_w4a4\",\n            \"nvfp4\",\n        ):\n            continue\n        missing = required - checkpoint_meta.keys()\n        if missing:\n            raise ValueError(\n                f\"Comfy layer {prefix!r} is missing checkpoint tensors: \"\n                f\"{sorted(missing)}\"\n            )\n        if marker_format == \"float8_e4m3fn\":\n            marker[\"_activation_scheme\"] = (\n                \"static\" if f\"{prefix}.input_scale\" in checkpoint_meta else \"dynamic\"\n            )\n            continue\n        if marker_format == \"asym_w4a8_int8\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_s_rel\"]\n            channel_dtype, channel_shape = checkpoint_meta[f\"{prefix}.weight_s_channel\"]\n            group_size = int(marker.get(\"group_size\", 16))\n            if group_size < 4:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}\"\n                )\n            if weight_dtype != \"I8\" or scale_dtype != \"F8_E4M3\":","sourceCodeStart":140,"sourceCodeEnd":176,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L140-L176","documentation":"Raised while inspecting a ComfyUI-style quantized checkpoint: the quant marker for a layer declares a required format, but the safetensors checkpoint lacks one or more tensors the format requires (weight, scales, etc.). The library validates checkpoint completeness before building an encoder quant config, so a partial or mismatched export is rejected early.","triggerScenarios":"Calling inspect_comfy_quant_markers (directly or via _get_encoder_quant_config / inspect_minimax_h3_safetensors) on a checkpoint where a tensor prefix has a quant marker whose format requires keys like {prefix}.weight, {prefix}.weight_s_rel, etc., but at least one required key is absent from the safetensors metadata.","commonSituations":"Checkpoint exported with a partial quantization pass (some layers skipped mid-export), safetensors file split into shards and only one shard loaded, manual pruning/renaming of tensors, or a version change in the export tool that renamed required tensor keys.","solutions":["Inspect the safetensors file (e.g. safetensors.safe_open(...).keys()) and compare against the marker's required tensor names for the failing prefix","Re-export or re-download the checkpoint so every layer carrying a quant marker also has all its required tensors","If the layer was intentionally left unquantized, remove its quant marker from the checkpoint","If loading sharded weights, ensure all shards are passed to the inspector"],"exampleFix":"# before: partial checkpoint\nload_safetensors(\"model.safetensors\")  # raises ValueError: missing checkpoint tensors\n\n# after: verify required tensors exist first\nmeta = read_safetensors_meta(\"model.safetensors\")\nassert all(k in meta for k in required_tensor_names(prefix)), 'incomplete export'\nload_safetensors(\"model.safetensors\")","handlingStrategy":"validation","validationCode":"from safetensors import safe_open\n\nrequired = {f\"{prefix}.{suffix}\" for suffix in REQUIRED_SUFFIXES[marker_format]}\nwith safe_open(path, framework=\"np\") as f:\n    keys = set(f.keys())\nmissing = required - keys\nif missing:\n    raise FileNotFoundError(f\"incomplete checkpoint, missing {sorted(missing)}\")","typeGuard":"def is_complete_comfy_checkpoint(meta: dict, markers: dict) -> bool:\n    return all(\n        REQUIRED_SUFFIXES.get(m.get(\"format\"), ()) <= {\n            k.removeprefix(p + \".\") for k in meta if k.startswith(p + \".\")\n        }\n        for p, m in markers.items()\n    )","tryCatchPattern":null,"preventionTips":["Run a tensor-key audit over safetensors metadata before loading","Keep quant markers and tensors in the same export pass","When sharding, validate all shards are present before inspection"],"tags":["quantization","checkpoint","safetensors","validation"],"backgroundTag":"checkpoint-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}