{"record":{"id":"178c581eb1ed4a34","repo":"deepseek-ai/DeepSeek-V3","slug":"dimension-dim-must-be-divisible-by-mp","errorCode":null,"errorMessage":"Dimension ${dim} must be divisible by ${mp}","messagePattern":"Dimension (.+?) must be divisible by (.+?)","errorType":"validation","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"inference/convert.py","lineNumber":73,"sourceCode":"                param: torch.Tensor = f.get_tensor(name)\n                if name.startswith(\"model.\"):\n                    name = name[len(\"model.\"):]\n                name = name.replace(\"self_attn\", \"attn\")\n                name = name.replace(\"mlp\", \"ffn\")\n                name = name.replace(\"weight_scale_inv\", \"scale\")\n                name = name.replace(\"e_score_correction_bias\", \"bias\")\n                key = name.split(\".\")[-2]\n                assert key in mapping, f\"Key {key} not found in mapping\"\n                new_key, dim = mapping[key]\n                name = name.replace(key, new_key)\n                for i in range(mp):\n                    new_param = param\n                    if \"experts\" in name and \"shared_experts\" not in name:\n                        idx = int(name.split(\".\")[-3])\n                        if idx < i * n_local_experts or idx >= (i + 1) * n_local_experts:\n                            continue\n                    elif dim is not None:\n                        assert param.size(dim) % mp == 0, f\"Dimension {dim} must be divisible by {mp}\"\n                        shard_size = param.size(dim) // mp\n                        new_param = param.narrow(dim, i * shard_size, shard_size).contiguous()\n                    state_dicts[i][name] = new_param\n\n    os.makedirs(save_path, exist_ok=True)\n\n    for i in trange(mp):\n        save_file(state_dicts[i], os.path.join(save_path, f\"model{i}-mp{mp}.safetensors\"))\n\n    for file_path in glob(os.path.join(hf_ckpt_path, \"*token*\")):\n        new_file_path = os.path.join(save_path, os.path.basename(file_path))\n        shutil.copyfile(file_path, new_file_path)\n\n\nif __name__ == \"__main__\":\n    parser = ArgumentParser()\n    parser.add_argument(\"--hf-ckpt-path\", type=str, required=True)\n    parser.add_argument(\"--save-path\", type=str, required=True)","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/convert.py#L55-L91","documentation":"Thrown in convert.py (inference/convert.py:73): for non-expert weights whose mapping entry declares a sharding dimension, the checkpoint tensor must split evenly into mp shards (param.narrow(dim, i*shard, shard)). This enforces that the HF checkpoint's linear dimensions are compatible with the chosen --model-parallel degree before writing model{i}-mp{mp}.safetensors.","triggerScenarios":"Running convert.py --model-parallel N where N does not divide the tensor's sharded dimension — e.g. mp=3 against hidden 7168, or a checkpoint whose q_proj output dim (e.g. 24576... not divisible by the chosen mp) mismatches. Also fires when converting a checkpoint with non-standard dims.","commonSituations":"Choosing mp based on target GPUs (e.g. 6) without checking DeepSeek-V3 dims; converting a distilled/variant checkpoint with odd dims; passing --model-parallel inconsistent with the world_size you will launch later (which would then hit errors 1/2 at runtime).","solutions":["Use --model-parallel from {2,4,8,16} (divisors of all DeepSeek-V3 linear dims)","Match --model-parallel exactly to the GPU count / world_size you will run inference with","If converting a variant model, pre-check each mapping shard dim: print(name, dim, param.size(dim), param.size(dim) % mp)"],"exampleFix":"# before\npython convert.py --hf-ckpt-path hf/ --save-path out/ --n-experts 256 --model-parallel 6\n# AssertionError: Dimension 0 must be divisible by 6\n\n# after\npython convert.py --hf-ckpt-path hf/ --save-path out/ --n-experts 256 --model-parallel 8","handlingStrategy":"validation","validationCode":"mp = 8  # planned\nVOCAB, HIDDEN = 129280, 7168\nfor label, d in [(\"vocab\", VOCAB), (\"hidden\", HIDDEN), (\"moe_inter\", 2048)]:\n    assert d % mp == 0, f\"{label}={d} not divisible by mp={mp}\"","typeGuard":"def shardable(size: int, mp: int) -> bool:\n    return size % mp == 0","tryCatchPattern":null,"preventionTips":["Choose --model-parallel from {2,4,8,16} for DeepSeek-V3","Keep conversion mp identical to the inference world_size","Pre-check every mapped shard dim with a small script before the hours-long conversion"],"tags":["checkpoint-conversion","model-parallelism","sharding","validation","deepseek"],"backgroundTag":null,"analyzedSha":"9b4e9788e4a3a731f7567338ed15d3ec549ce03b","analyzedAt":"2026-08-14T19:02:32.748Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}