invoke-ai/InvokeAI · info · NotAMatchError

state dict looks like a Gemma-2 encoder (has post_attention_

Error message

state dict looks like a Gemma-2 encoder (has post_attention_norm/post_ffw_norm keys), not a Qwen3 encoder

What it means

NotAMatchError raised by Qwen3Encoder._validate_looks_like_qwen3_model (qwen3_encoder.py:242). Gemma-2/3 GGUF encoders share token_embd.weight + blk.* keys with Qwen3 but additionally have post_attention_norm/post_ffw_norm tensors, which Qwen3 never has. The config raises so the model is classified as Gemma2Encoder instead of being wrongly re-identified as a Qwen3 encoder.

Source

Thrown at invokeai/backend/model_manager/configs/qwen3_encoder.py:242

        variant = _get_qwen3_variant_from_state_dict(state_dict)
        if variant is None:
            raise NotAMatchError("hidden size does not match a known Qwen3 variant")
        return variant

    @classmethod
    def _validate_looks_like_qwen3_model(cls, mod: ModelOnDisk) -> None:
        state_dict = mod.load_state_dict()
        if not _has_qwen3_keys(state_dict):
            raise NotAMatchError("state dict does not look like a Qwen3 model")
        # Reject T5 encoders: they share the token_embd.weight key with Qwen3 GGUFs but use the ``enc.``
        # block prefix, and must be classified as T5Encoder (Qwen3 encoders never have ``enc.blk.*`` keys).
        if _has_t5_encoder_keys(state_dict):
            raise NotAMatchError("state dict looks like a T5 encoder (has 'enc.blk.*' keys), not a Qwen3 encoder")
        # Reject Gemma-2/3 encoders: their GGUFs also carry token_embd.weight + blk.* keys but use
        # post-attention / post-feedforward norms a Qwen3 encoder never has; they must be classified as
        # Gemma2Encoder (otherwise a Gemma GGUF matches both configs and can be re-identified wrongly).
        if _has_gemma2_keys(state_dict):
            raise NotAMatchError(
                "state dict looks like a Gemma-2 encoder (has post_attention_norm/post_ffw_norm keys), "
                "not a Qwen3 encoder"
            )
        # Reject Qwen2.5-VL / Qwen2-VL encoders: they carry a visual tower and must be
        # classified as QwenVLEncoder (text-only Qwen3 encoders never have one).
        if _has_qwen_vl_visual_tower(state_dict):
            raise NotAMatchError(
                "state dict bundles a Qwen-VL visual tower; this is a Qwen-VL encoder, not a text-only Qwen3 encoder"
            )

    @classmethod
    def _validate_does_not_look_like_gguf_quantized(cls, mod: ModelOnDisk) -> None:
        has_ggml = _has_ggml_tensors(mod.load_state_dict())
        if has_ggml:
            raise NotAMatchError("state dict looks like GGUF quantized")


# Transformers architectures the unquantized Qwen3 encoder config accepts.

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Expected behavior during disambiguation — allow the scan to continue so Gemma2Encoder matches.
2. If the model ends up unidentified, install it explicitly as a Gemma2Encoder model.
3. Verify the GGUF is truly Gemma (check metadata/general.architecture = gemma); if it's actually Qwen3 with stray keys, re-convert it.
4. Keep the Gemma encoder in its own folder so it is not probed as a Qwen3 encoder candidate.

Example fix

// before
models/encoders/gemma-3-encoder.gguf  // inside qwen3 scan path
// after
models/encoders/gemma-3-encoder.gguf installed as Gemma2Encoder (or moved out of the Qwen3 encoder folder)

def is_gemma_encoder(state_dict: dict) -> bool:
    keys = set(state_dict)
    return any('post_attention_norm' in k or 'post_ffw_norm' in k for k in keys)  # install as Gemma2Encoder

def is_qwen3_not_gemma(state_dict: dict) -> bool:
    keys = set(state_dict)
    has_norm_keys = any('post_attention_norm' in k or 'post_ffw_norm' in k for k in keys)
    return (any(k.startswith('blk.') for k in keys) or 'token_embd.weight' in keys) and not has_norm_keys

try:
    register_model(path, model_type='Qwen3Encoder')
except NotAMatchError:
    register_model(path, model_type='Gemma2Encoder')

When it happens

Trigger: from_model_on_disk identification of a Gemma-2/Gemma-3 text-encoder GGUF whose state dict contains post_attention_norm/post_ffw_norm keys while the Qwen3Encoder config probes it.

Common situations: Installing a Gemma 2 / Gemma 3 GGUF text encoder (e.g. for FLUX or other pipelines) into InvokeAI; key-collision between llama.cpp tensor naming conventions across Gemma and Qwen3.

Related errors

• hidden size does not match a known Qwen3 variant
• state dict looks like a T5 encoder (has 'enc.blk.*' keys), n
• state dict does not look like a Wan transformer
• state dict has no undecorated transformer block weights — it
• state dict does not look like a Qwen3 model

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/f7a3d34cb6504796. Report an issue: GitHub.