What causes "Dimension ${dim} must be divisible by ${mp}" in deepseek-ai/DeepSeek-V3?

Use --model-parallel from {2,4,8,16} (divisors of all DeepSeek-V3 linear dims)

deepseek-ai/DeepSeek-V3 · error · AssertionError

Dimension ${dim} must be divisible by ${mp}

Error message

Dimension ${dim} must be divisible by ${mp}

What it means

Thrown in convert.py (inference/convert.py:73): for non-expert weights whose mapping entry declares a sharding dimension, the checkpoint tensor must split evenly into mp shards (param.narrow(dim, i*shard, shard)). This enforces that the HF checkpoint's linear dimensions are compatible with the chosen --model-parallel degree before writing model{i}-mp{mp}.safetensors.

Source

Thrown at inference/convert.py:73

                param: torch.Tensor = f.get_tensor(name)
                if name.startswith("model."):
                    name = name[len("model."):]
                name = name.replace("self_attn", "attn")
                name = name.replace("mlp", "ffn")
                name = name.replace("weight_scale_inv", "scale")
                name = name.replace("e_score_correction_bias", "bias")
                key = name.split(".")[-2]
                assert key in mapping, f"Key {key} not found in mapping"
                new_key, dim = mapping[key]
                name = name.replace(key, new_key)
                for i in range(mp):
                    new_param = param
                    if "experts" in name and "shared_experts" not in name:
                        idx = int(name.split(".")[-3])
                        if idx < i * n_local_experts or idx >= (i + 1) * n_local_experts:
                            continue
                    elif dim is not None:
                        assert param.size(dim) % mp == 0, f"Dimension {dim} must be divisible by {mp}"
                        shard_size = param.size(dim) // mp
                        new_param = param.narrow(dim, i * shard_size, shard_size).contiguous()
                    state_dicts[i][name] = new_param

    os.makedirs(save_path, exist_ok=True)

    for i in trange(mp):
        save_file(state_dicts[i], os.path.join(save_path, f"model{i}-mp{mp}.safetensors"))

    for file_path in glob(os.path.join(hf_ckpt_path, "*token*")):
        new_file_path = os.path.join(save_path, os.path.basename(file_path))
        shutil.copyfile(file_path, new_file_path)


if __name__ == "__main__":
    parser = ArgumentParser()
    parser.add_argument("--hf-ckpt-path", type=str, required=True)
    parser.add_argument("--save-path", type=str, required=True)

View on GitHub (pinned to 9b4e9788e4)

Solutions

Use --model-parallel from {2,4,8,16} (divisors of all DeepSeek-V3 linear dims)
Match --model-parallel exactly to the GPU count / world_size you will run inference with
If converting a variant model, pre-check each mapping shard dim: print(name, dim, param.size(dim), param.size(dim) % mp)

Example fix

# before
python convert.py --hf-ckpt-path hf/ --save-path out/ --n-experts 256 --model-parallel 6
# AssertionError: Dimension 0 must be divisible by 6

# after
python convert.py --hf-ckpt-path hf/ --save-path out/ --n-experts 256 --model-parallel 8

Defensive patterns

Strategy: validation

Validate before calling

mp = 8  # planned
VOCAB, HIDDEN = 129280, 7168
for label, d in [("vocab", VOCAB), ("hidden", HIDDEN), ("moe_inter", 2048)]:
    assert d % mp == 0, f"{label}={d} not divisible by mp={mp}"

Type guard

def shardable(size: int, mp: int) -> bool:
    return size % mp == 0

Prevention

Choose --model-parallel from {2,4,8,16} for DeepSeek-V3
Keep conversion mp identical to the inference world_size
Pre-check every mapped shard dim with a small script before the hours-long conversion

When it happens

Trigger: Running convert.py --model-parallel N where N does not divide the tensor's sharded dimension — e.g. mp=3 against hidden 7168, or a checkpoint whose q_proj output dim (e.g. 24576... not divisible by the chosen mp) mismatches. Also fires when converting a checkpoint with non-standard dims.

Common situations: Choosing mp based on target GPUs (e.g. 6) without checking DeepSeek-V3 dims; converting a distilled/variant checkpoint with odd dims; passing --model-parallel inconsistent with the world_size you will launch later (which would then hit errors 1/2 at runtime).

Related errors

AI-assisted analysis of deepseek-ai/DeepSeek-V3@9b4e9788e4 (2026-08-14). Data as JSON: /api/errors/178c581eb1ed4a34. Report an issue: GitHub.