invoke-ai/InvokeAI · error · TypeError

Expected PreTrainedModel for Gemma encoder, got {type(gemma_

Error message

Expected PreTrainedModel for Gemma encoder, got {type(gemma_encoder).__name__}.

What it means

z_image_pid_decode.py loads the Gemma text encoder via context.models and asserts the object returned by model_on_device() is a transformers PreTrainedModel before encoding. If the loader yields any other type (wrapper, stub, wrong module), a TypeError is raised identifying the actual class. This guards against a corrupted or incorrectly-imported Gemma encoder being fed to the PiD decode path.

Source

Thrown at invokeai/app/invocations/z_image_pid_decode.py:140

                    # FluxAutoEncoder stores the constants directly on the module.
                    scaling_factor = float(getattr(vae, "scale_factor", scaling_factor))
                    shift_factor = float(getattr(vae, "shift_factor", shift_factor))
            del vae_info
            TorchDevice.empty_cache()
        context.logger.info(
            f"Z-Image PiD decode: latent shape={tuple(latents.shape)} dtype={latents.dtype} "
            f"stats[min={latents.min().item():.3f} max={latents.max().item():.3f} "
            f"mean={latents.mean().item():.3f}] using scale={scaling_factor:.4f} shift={shift_factor:.4f}"
        )

        # 2) Encode caption with Gemma-2.
        gemma_text_encoder_info = context.models.load(self.gemma2_encoder.text_encoder)
        gemma_tokenizer_info = context.models.load(self.gemma2_encoder.tokenizer)
        with ExitStack() as stack:
            (_, gemma_encoder) = stack.enter_context(gemma_text_encoder_info.model_on_device())
            (_, gemma_tokenizer) = stack.enter_context(gemma_tokenizer_info.model_on_device())
            if not isinstance(gemma_encoder, PreTrainedModel):
                raise TypeError(f"Expected PreTrainedModel for Gemma encoder, got {type(gemma_encoder).__name__}.")
            if not isinstance(gemma_tokenizer, PreTrainedTokenizerBase):
                raise TypeError(
                    f"Expected PreTrainedTokenizerBase for Gemma tokenizer, got {type(gemma_tokenizer).__name__}."
                )

            # Encode on the encoder's intended compute device. compute_device honours cpu_only and is
            # stable under partial loading — the first parameter may be offloaded to CPU while later
            # modules load on CUDA, so inferring the device from the first parameter could place caption
            # inputs on the wrong device.
            device = gemma_text_encoder_info.compute_device
            encode_dtype = TorchDevice.choose_bfloat16_safe_dtype(device)

            context.util.signal_progress("Encoding caption with Gemma-2")
            caption_embs, caption_mask = encode_caption_for_pid(
                [self.prompt],
                tokenizer=gemma_tokenizer,
                encoder=gemma_encoder,
                device=device,

View on GitHub (pinned to 0b6a024f2f)

Solutions

Re-download or re-import the Gemma text encoder so it is a standard transformers PreTrainedModel.
Verify the gemma2_encoder submodel points at the correct model config in the model manager.
Update InvokeAI and transformers to compatible versions so loaded models resolve to PreTrainedModel.
Check the loaded model files (config.json/model weights) for corruption and repair with the model installer.

Defensive patterns

Strategy: type-guard

Validate before calling

info = context.models.load(gemma_encoder_key)
with info.model_on_device() as (_, enc):
    if not isinstance(enc, PreTrainedModel):
        fail_fast(enc)

Type guard

def is_pretrained_model(obj) -> bool:
    from transformers import PreTrainedModel
    return isinstance(obj, PreTrainedModel)

Try / catch

try:
    decode(context)
except TypeError as e:
    if "Expected PreTrainedModel for Gemma encoder" in str(e):
        reinstall_gemma_encoder()
    else:
        raise

Prevention

Install Gemma encoder only via the model manager so it loads as a standard transformers model.
Verify checksums after downloading to avoid corrupted encoder files.
Keep transformers and InvokeAI versions in sync.

When it happens

Trigger: Calling the PiD decode invocation whose gemma2_encoder submodel loads to an object that fails isinstance(gemma_encoder, PreTrainedModel) — i.e. model_on_device returns a non-transformers model object.

Common situations: Corrupted or partially downloaded Gemma encoder files; a Gemma text-encoder model imported with an incompatible format or converted by a tool producing a non-PreTrainedModel wrapper; transformers/InvokeAI version mismatch in model loading classes.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/aaab4371e4bffccc. Report an issue: GitHub.