{"record":{"id":"996a3ef3448d0bbb","repo":"deepseek-ai/DeepSeek-V3","slug":"warning-missing-scale-inv-tensor-for-weight-nam","errorCode":null,"errorMessage":"Warning: Missing scale_inv tensor for ${weight_name}, skipping conversion","messagePattern":"Warning: Missing scale_inv tensor for (.+?), skipping conversion","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"inference/fp8_cast_bf16.py","lineNumber":82,"sourceCode":"    safetensor_files.sort()\n    for safetensor_file in tqdm(safetensor_files):\n        file_name = os.path.basename(safetensor_file)\n        current_state_dict = load_file(safetensor_file, device=\"cuda\")\n        loaded_files[file_name] = current_state_dict\n        \n        new_state_dict = {}\n        for weight_name, weight in current_state_dict.items():\n            if weight_name.endswith(\"_scale_inv\"):\n                continue\n            elif weight.element_size() == 1:  # FP8 weight\n                scale_inv_name = f\"{weight_name}_scale_inv\"\n                try:\n                    # Get scale_inv from the correct file\n                    scale_inv = get_tensor(scale_inv_name)\n                    fp8_weight_names.append(weight_name)\n                    new_state_dict[weight_name] = weight_dequant(weight, scale_inv)\n                except KeyError:\n                    print(f\"Warning: Missing scale_inv tensor for {weight_name}, skipping conversion\")\n                    new_state_dict[weight_name] = weight\n            else:\n                new_state_dict[weight_name] = weight\n                \n        new_safetensor_file = os.path.join(bf16_path, file_name)\n        save_file(new_state_dict, new_safetensor_file)\n        \n        # Memory management: keep only the 2 most recently used files\n        if len(loaded_files) > 2:\n            oldest_file = next(iter(loaded_files))\n            del loaded_files[oldest_file]\n            torch.cuda.empty_cache()\n    \n    # Update model index\n    new_model_index_file = os.path.join(bf16_path, \"model.safetensors.index.json\")\n    for weight_name in fp8_weight_names:\n        scale_inv_name = f\"{weight_name}_scale_inv\"\n        if scale_inv_name in weight_map:","sourceCodeStart":64,"sourceCodeEnd":100,"githubUrl":"https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/fp8_cast_bf16.py#L64-L100","documentation":"This is a printed WARNING (not an exception) from fp8_cast_bf16.py:82: a weight with element_size()==1 (FP8) had no matching '{weight}_scale_inv' tensor findable via get_tensor (which raises KeyError when the name is absent from weight_map or from the loaded file). The except KeyError path keeps the raw FP8 bytes in the BF16 output and continues — downstream code then silently loads a mixed-precision checkpoint, which will produce garbage activations.","triggerScenarios":"Running fp8_cast_bf16.py on a checkpoint where: the scale tensor is named differently (e.g. 'weight_scale' without _inv), the model.safetensors.index.json weight_map lacks the scale entry (corrupt/partial download), or the scale lives in a file whose name differs from the map. The per-file iteration reads via load_file (KeyError on missing key) and get_tensor (KeyError on missing weight_map entry).","commonSituations":"Incomplete interrupted downloads from HF hub (missing shard scale entries); checkpoints from quantizers other than the official FP8 recipe (e.g. compressed-tensors uses weight_scale/axis-ep names); index.json from a different revision than the shard files.","solutions":["Re-verify the download: compare model.safetensors.index.json weight_map against actual files; re-download incomplete shards (huggingface-cli download --resume)","Check the actual scale naming in the checkpoint and, if different (e.g. ...scale or ...scale_weight), adjust scale_inv_name or pre-rename keys","Treat this warning as fatal for correctness — grep the output for 'Missing scale_inv' and fail the run instead of shipping mixed FP8/BF16 weights","Confirm you are converting the official DeepSeek-V3/R1 FP8 checkpoint that this script was written for"],"exampleFix":"# before (fp8_cast_bf16.py)\nexcept KeyError:\n    print(f\"Warning: Missing scale_inv tensor for {weight_name}, skipping conversion\")\n    new_state_dict[weight_name] = weight\n\n# after — fail fast on missing scales\nexcept KeyError:\n    raise RuntimeError(\n        f\"Missing scale_inv for {weight_name}: FP8 checkpoint is incomplete or \"\n        f\"uses different scale naming; re-download or fix the index\"\n    )","handlingStrategy":"try-catch","validationCode":"import json, os\nfrom glob import glob\n\nfp8_path = \"hf/deepseek-ai/DeepSeek-V3\"\nwith open(os.path.join(fp8_path, \"model.safetensors.index.json\")) as f:\n    weight_map = json.load(f)[\"weight_map\"]\n\n# Every FP8 weight must have a scale_inv entry mapped to an existing file\nall_keys = set()\nfor shard in glob(os.path.join(fp8_path, \"*.safetensors\")):\n    from safetensors import safe_open\n    with safe_open(shard, framework=\"pt\") as sf:\n        all_keys.update(sf.keys())\nmissing = [k for k in all_keys\n           if not k.endswith(\"_scale_inv\") and f\"{k}_scale_inv\" not in all_keys]\n# element_size check happens at runtime; approximate by name for FP8 checkpoints\nassert not missing, f\"weights missing scale_inv (incomplete download?): {missing[:5]}\"","typeGuard":"def has_scale_inv(weight_name: str, weight_map: dict, loaded_keys: set) -> bool:\n    scale = f\"{weight_name}_scale_inv\"\n    return scale in weight_map and scale in loaded_keys","tryCatchPattern":"# Fail fast instead of printing a warning and emitting mixed-precision weights\ntry:\n    scale_inv = get_tensor(scale_inv_name)\nexcept KeyError:\n    raise RuntimeError(\n        f\"Missing {scale_inv_name}: the FP8 checkpoint is incomplete or its scale \"\n        f\"tensors use different naming. Re-download the checkpoint or fix the index; \"\n        f\"continuing would silently keep FP8 bytes in the BF16 output.\"\n    )","preventionTips":["Grep conversion logs for 'Missing scale_inv' and treat any hit as a failed run","Verify download completeness with huggingface-cli download (resumable) and check index.json entry counts","Only feed this script the official DeepSeek FP8 checkpoints; other quantization schemes name scales differently","After conversion, sanity-check output dtypes: no tensor in the BF16 output should have element_size()==1"],"tags":["fp8","dequantization","checkpoint-conversion","safetensors","silent-corruption","deepseek"],"backgroundTag":null,"analyzedSha":"9b4e9788e4a3a731f7567338ed15d3ec549ce03b","analyzedAt":"2026-08-14T19:02:32.748Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}