{"record":{"id":"5ff295ca4bc82980","repo":"invoke-ai/InvokeAI","slug":"gemma2-gguf-embedding-length-hidden-size-is-inco","errorCode":null,"errorMessage":"Gemma2 GGUF embedding_length {hidden_size} is incompatible with PiD, which requires {_PID_GEMMA_HIDDEN_SIZE} (Gemma-2-2b); 9B/27B variants are not supported.","messagePattern":"Gemma2 GGUF embedding_length (.+?) is incompatible with PiD, which requires (.+?) \\(Gemma-2-2b\\); 9B/27B variants are not supported\\.","errorType":"validation","errorClass":"NotAMatchError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/model_manager/configs/gemma2_encoder.py","lineNumber":140,"sourceCode":"\n    base: Literal[BaseModelType.Any] = Field(default=BaseModelType.Any)\n    type: Literal[ModelType.Gemma2Encoder] = Field(default=ModelType.Gemma2Encoder)\n    format: Literal[ModelFormat.GGUFQuantized] = Field(default=ModelFormat.GGUFQuantized)\n    cpu_only: bool | None = Field(default=None, description=\"Whether this model should run on CPU only\")\n\n    @classmethod\n    def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self:\n        raise_if_not_file(mod)\n        raise_for_override_fields(cls, override_fields)\n\n        if mod.path.suffix.lower() != \".gguf\":\n            raise NotAMatchError(f\"not a .gguf file: {mod.path.name}\")\n\n        architecture, hidden_size = _read_gguf_arch_and_hidden_size(mod.path)\n        if architecture != \"gemma2\":\n            raise NotAMatchError(f\"GGUF architecture '{architecture}' is not 'gemma2'\")\n        if hidden_size != _PID_GEMMA_HIDDEN_SIZE:\n            raise NotAMatchError(\n                f\"Gemma2 GGUF embedding_length {hidden_size} is incompatible with PiD, which requires \"\n                f\"{_PID_GEMMA_HIDDEN_SIZE} (Gemma-2-2b); 9B/27B variants are not supported.\"\n            )\n\n        return cls(**override_fields)\n","sourceCodeStart":122,"sourceCodeEnd":146,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/model_manager/configs/gemma2_encoder.py#L122-L146","documentation":"NotAMatchError raised by Gemma2Encoder_GGUF_Config.from_model_on_disk when a gemma2-architecture GGUF reports an embedding_length other than 2304. PiD's caption projection is fixed to Gemma-2-2b's 2304-dim hidden state, so 9B (3584) and 27B (4608) GGUFs are rejected at import time instead of failing later with a matrix-shape error during inference.","triggerScenarios":"from_model_on_disk on a .gguf with general.architecture=='gemma2' whose <arch>.embedding_length != 2304 — i.e. gemma-2-9b-it or gemma-2-27b-it quantizations.","commonSituations":"User picks the largest GGUF in a bartowski-style collection assuming bigger is better; converts gemma-2-9b/27b to GGUF themselves and tries to use it as the PiD encoder.","solutions":["Use a Gemma-2-2b GGUF (embedding_length 2304), e.g. gemma-2-2b-it-Q4_K_M.gguf","Check before importing: gguf-dump file.gguf | grep embedding_length","Verify the GGUF was built from gemma-2-2b-it, not the 9b/27b checkpoints"],"exampleFix":"// before (inspect)\n<gemma2>.embedding_length = 3584  # gemma-2-9b-it GGUF\n// after\n<gemma2>.embedding_length = 2304  # gemma-2-2b-it GGUF","handlingStrategy":"validation","validationCode":"def gguf_embedding_length(path) -> int | None:\n    import gguf\n    try:\n        reader = gguf.GGUFReader(path)\n        field = reader.fields.get(\"general.architecture\")\n        if field is None:\n            return None\n        arch = str(field.contents())\n        hidden = reader.fields.get(f\"{arch}.embedding_length\")\n        return int(hidden.contents()) if hidden else None\n    except Exception:\n        return None\n\nassert gguf_embedding_length(\"model.gguf\") == 2304","typeGuard":"def is_gemma_2_2b_gguf(p: Path) -> bool:\n    return gguf_architecture(p) == \"gemma2\" and gguf_embedding_length(p) == 2304","tryCatchPattern":"try:\n    import_model(gguf_path)\nexcept NotAMatchError as e:\n    if \"embedding_length\" in str(e):\n        print(\"9B/27B GGUF not supported by PiD — use gemma-2-2b-it (2304-dim)\")","preventionTips":["Confirm embedding_length == 2304 via gguf-dump before adding any gemma2 GGUF as a PiD encoder","In quantization collections, pick files explicitly named gemma-2-2b-it-*","Don't assume quantization level changes compatibility — it's the base model size that matters"],"tags":["gguf","model-compatibility","gemma2"],"backgroundTag":"model-variant-incompatible","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}