{"record":{"id":"681985fdc720b875","repo":"invoke-ai/InvokeAI","slug":"expected-2d-embed-tokens-weight-tensor-got-shape-681985","errorCode":null,"errorMessage":"Expected 2D embed_tokens weight tensor, got shape {embed_shape}.","messagePattern":"Expected 2D embed_tokens weight tensor, got shape (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/model_manager/load/model_loaders/z_image.py","lineNumber":1532,"sourceCode":"        layer_count = 0\n        for key in sd.keys():\n            if isinstance(key, str) and key.startswith(\"model.layers.\"):\n                parts = key.split(\".\")\n                if len(parts) > 2:\n                    try:\n                        layer_idx = int(parts[2])\n                        layer_count = max(layer_count, layer_idx + 1)\n                    except ValueError:\n                        pass\n\n        # Get hidden size from embed_tokens weight shape\n        embed_weight = sd.get(\"model.embed_tokens.weight\")\n        if embed_weight is None:\n            raise ValueError(\"Could not find model.embed_tokens.weight in state dict\")\n\n        embed_shape = embed_weight.shape if hasattr(embed_weight, \"shape\") else embed_weight.tensor_shape\n        if len(embed_shape) != 2:\n            raise ValueError(f\"Expected 2D embed_tokens weight tensor, got shape {embed_shape}.\")\n        hidden_size = embed_shape[1]\n        vocab_size = embed_shape[0]\n\n        # Detect attention configuration from layer 0 weights\n        q_proj_weight = sd.get(\"model.layers.0.self_attn.q_proj.weight\")\n        k_proj_weight = sd.get(\"model.layers.0.self_attn.k_proj.weight\")\n        gate_proj_weight = sd.get(\"model.layers.0.mlp.gate_proj.weight\")\n\n        if q_proj_weight is None or k_proj_weight is None or gate_proj_weight is None:\n            raise ValueError(\"Could not find attention/mlp weights in state dict to determine configuration\")\n\n        q_shape = q_proj_weight.shape if hasattr(q_proj_weight, \"shape\") else q_proj_weight.tensor_shape\n        k_shape = k_proj_weight.shape if hasattr(k_proj_weight, \"shape\") else k_proj_weight.tensor_shape\n        gate_shape = gate_proj_weight.shape if hasattr(gate_proj_weight, \"shape\") else gate_proj_weight.tensor_shape\n\n        head_dim = 128  # Standard head dimension for Qwen3 models\n        num_attention_heads = q_shape[0] // head_dim\n        num_kv_heads = k_shape[0] // head_dim","sourceCodeStart":1514,"sourceCodeEnd":1550,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/model_manager/load/model_loaders/z_image.py#L1514-L1550","documentation":"Companion to the missing-key error: 'model.embed_tokens.weight' was found but its shape is not 2D, so hidden_size/vocab_size cannot be derived from it. Indicates a malformed, corrupted, or unconventionally quantized SDNQ file.","triggerScenarios":"Loading an SDNQ model whose embedding tensor was flattened or packed to >2D by a custom quantization scheme, or whose file is truncated so tensor metadata is wrong.","commonSituations":"Custom/experimental SDNQ quant formats applied to the embedding layer; corrupted download; conversion bug in a third-party SDNQ exporter.","solutions":["Re-quantize the model keeping embed_tokens.weight in a standard 2D (bf16/f16) layout.","Re-download and verify file integrity.","Use a known-good SDNQ release of the Qwen3 encoder instead of a hand-converted one.","If a custom packing scheme is intentional, unpack/dequantize the tensor before this loader reads the state dict."],"exampleFix":"// before\n'model.embed_tokens.weight' shape [151669*2048] (1D packed)\n// after\n'model.embed_tokens.weight' shape [151669, 2048]","handlingStrategy":"validation","validationCode":"w = sd[\"model.embed_tokens.weight\"]\nshape = w.shape if hasattr(w, \"shape\") else w.tensor_shape\nif len(shape) != 2:\n    raise ValueError(f\"{path}: embed_tokens must be 2D, got {shape} — re-quantize with standard embedding layout\")","typeGuard":"def has_2d_embed(sd: dict) -> bool:\n    w = sd.get(\"model.embed_tokens.weight\")\n    if w is None:\n        return False\n    s = w.shape if hasattr(w, \"shape\") else getattr(w, \"tensor_shape\", None)\n    return s is not None and len(s) == 2","tryCatchPattern":"try:\n    model = load_text_encoder(cfg)\nexcept ValueError as e:\n    if \"Expected 2D embed_tokens\" in str(e):\n        raise ModelIntegrityError(f\"SDNQ file {cfg.path} malformed; use a known-good release.\") from e\n    raise","preventionTips":["Keep embed_tokens.weight unquantized/standard 2D when quantizing.","Validate tensor shapes right after any SDNQ conversion.","Prefer official SDNQ releases over custom-packed formats.","Hash-verify large quantized downloads."],"tags":["sdnq","shape-mismatch","corrupt-file","model-loading"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}