{"record":{"id":"af4a9228d1c6479b","repo":"invoke-ai/InvokeAI","slug":"expected-2d-embed-tokens-weight-tensor-got-shape","errorCode":null,"errorMessage":"Expected 2D embed_tokens weight tensor, got shape {embed_shape}. The model file may be corrupted or incompatible.","messagePattern":"Expected 2D embed_tokens weight tensor, got shape (.+?)\\. The model file may be corrupted or incompatible\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/model_manager/load/model_loaders/z_image.py","lineNumber":1243,"sourceCode":"        for key in sd.keys():\n            if isinstance(key, str) and key.startswith(\"model.layers.\"):\n                parts = key.split(\".\")\n                if len(parts) > 2:\n                    try:\n                        layer_idx = int(parts[2])\n                        layer_count = max(layer_count, layer_idx + 1)\n                    except ValueError:\n                        pass\n\n        # Get vocab size from embed_tokens weight shape\n        embed_weight = sd.get(\"model.embed_tokens.weight\")\n        if embed_weight is None:\n            raise ValueError(\"Could not find model.embed_tokens.weight in state dict\")\n\n        # Handle GGMLTensor shape access\n        embed_shape = embed_weight.shape if hasattr(embed_weight, \"shape\") else embed_weight.tensor_shape\n        if len(embed_shape) != 2:\n            raise ValueError(\n                f\"Expected 2D embed_tokens weight tensor, got shape {embed_shape}. \"\n                \"The model file may be corrupted or incompatible.\"\n            )\n        vocab_size = embed_shape[0]\n\n        # Detect attention configuration from layer weights\n        # IMPORTANT: Use layer 1 (not layer 0) because some models like FLUX 2 Klein have a special\n        # first layer with different dimensions (input projection layer) while the rest of the\n        # transformer layers have a different hidden_size. Using a middle layer ensures we get\n        # the representative hidden_size for the bulk of the model.\n        # Fall back to layer 0 if layer 1 doesn't exist.\n        q_proj_weight = sd.get(\"model.layers.1.self_attn.q_proj.weight\")\n        k_proj_weight = sd.get(\"model.layers.1.self_attn.k_proj.weight\")\n        gate_proj_weight = sd.get(\"model.layers.1.mlp.gate_proj.weight\")\n\n        # Fall back to layer 0 if layer 1 doesn't exist (single-layer model edge case)\n        if q_proj_weight is None:\n            q_proj_weight = sd.get(\"model.layers.0.self_attn.q_proj.weight\")","sourceCodeStart":1225,"sourceCodeEnd":1261,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/model_manager/load/model_loaders/z_image.py#L1225-L1261","documentation":"Raised when 'model.embed_tokens.weight' exists in the GGUF state dict but is not a 2D (vocab, hidden) tensor. Because GGUF tensors carry GGML shapes, the loader reads .shape (or .tensor_shape) to derive vocab_size; a malformed or quantized-off embedding tensor yields an unexpected rank, so the file is treated as corrupted/incompatible.","triggerScenarios":"Loading a GGUF text encoder where the embed_tokens tensor was flattened (1D), stored transposed with extra dims (3D), or quantized to a format whose metadata loses the 2D shape during conversion.","commonSituations":"Hand-converted GGUF from safetensors with a broken conversion script; corrupted download where tensor metadata is garbled; incompatible model generation whose embed tensor was fused or reshaped.","solutions":["Re-download or re-convert the GGUF with an up-to-date converter that preserves 2D embedding shapes.","Verify the tensor rank with gguf-dump; if it is 1D/3D, re-export from the original safetensors checkpoint.","If quantization packs the tensor, load a F16/FP32 variant of the text-encoder GGUF instead.","Check disk/full-download health: a truncated file can corrupt tensor headers producing wrong shapes."],"exampleFix":"// before: hand-rolled conversion flattened embeddings\ntensor 'model.embed_tokens.weight' shape [2315296]\n// after: re-convert preserving rank\ntensor 'model.embed_tokens.weight' shape [151669, 2048]","handlingStrategy":"validation","validationCode":"import gguf\nreader = gguf.GGUFReader(path)\nfor t in reader.tensors:\n    if t.name == \"model.embed_tokens.weight\":\n        shape = t.shape\n        if shape is None or len(shape) != 2:\n            raise ValueError(f\"Bad embed shape {shape} in {path}; re-convert the GGUF.\")","typeGuard":"def has_2d_embedding(path: str) -> bool:\n    try:\n        import gguf\n        for t in gguf.GGUFReader(path).tensors:\n            if t.name == \"model.embed_tokens.weight\":\n                return len(t.shape) == 2\n    except Exception:\n        pass\n    return False","tryCatchPattern":"try:\n    model = load_text_encoder(cfg)\nexcept ValueError as e:\n    if \"Expected 2D embed_tokens\" in str(e):\n        raise ModelValidationError(f\"GGUF {cfg.path} corrupt; re-download/re-convert.\") from e\n    raise","preventionTips":["Only use GGUFs converted with maintained converter versions.","Sanity-check shapes via gguf-dump after conversion.","Prefer official F16 releases over hand-quantized files.","Verify downloads complete (size + hash)."],"tags":["gguf","model-loading","shape-mismatch","corrupt-file"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}