invoke-ai/InvokeAI · error · RuntimeError

Selected CLIP Vision Model is incompatible with the current

Error message

Selected CLIP Vision Model is incompatible with the current IP Adapter

What it means

IPAdapter.get_image_embeds runs the CLIP vision encoder output through the adapter's image projection model (_image_proj_model). If that forward pass raises a RuntimeError (typically a torch matmul/shape mismatch), the library re-raises with this message because it means the image encoder's embedding dimension does not match what the IP-Adapter weights were trained for.

Source

Thrown at invokeai/backend/ip_adapter/ip_adapter.py:156

        from invokeai.backend.model_manager.load.model_util import calc_module_size

        return calc_module_size(self._image_proj_model) + calc_module_size(self.attn_weights)

    def _init_image_proj_model(
        self, state_dict: dict[str, torch.Tensor]
    ) -> Union[ImageProjModel, Resampler, MLPProjModel]:
        return ImageProjModel.from_state_dict(state_dict, self._num_tokens).to(self.device, dtype=self.dtype)

    @torch.inference_mode()
    def get_image_embeds(self, pil_image: List[Image.Image], image_encoder: CLIPVisionModelWithProjection):
        clip_image = self._clip_image_processor(images=pil_image, return_tensors="pt").pixel_values
        clip_image_embeds = image_encoder(clip_image.to(self.device, dtype=self.dtype)).image_embeds
        try:
            image_prompt_embeds = self._image_proj_model(clip_image_embeds)
            uncond_image_prompt_embeds = self._image_proj_model(torch.zeros_like(clip_image_embeds))
            return image_prompt_embeds, uncond_image_prompt_embeds
        except RuntimeError as e:
            raise RuntimeError("Selected CLIP Vision Model is incompatible with the current IP Adapter") from e


class IPAdapterPlus(IPAdapter):
    """IP-Adapter with fine-grained features"""

    def _init_image_proj_model(self, state_dict: dict[str, torch.Tensor]) -> Union[Resampler, MLPProjModel]:
        return Resampler.from_state_dict(
            state_dict=state_dict,
            depth=4,
            dim_head=64,
            heads=12,
            num_queries=self._num_tokens,
            ff_mult=4,
        ).to(self.device, dtype=self.dtype)

    @torch.inference_mode()
    def get_image_embeds(self, pil_image: List[Image.Image], image_encoder: CLIPVisionModelWithProjection):
        clip_image = self._clip_image_processor(images=pil_image, return_tensors="pt").pixel_values

View on GitHub (pinned to 0b6a024f2f)

Solutions

Use the image encoder paired with the IP-Adapter checkpoint: SD1.5 IP-Adapters require 'h94/IP-Adapter' ViT-H (models/image_encoder/model.safetensors); SDXL IP-Adapter uses ViT-H as well, while some SDXL adapters use ViT-bigG — check the checkpoint's README.
Verify the IP-Adapter variant matches (IPAdapter vs IPAdapterPlus vs IPAdapterFull) — each initializes a different projection model.
Check the embedding dimension mismatch in the chained RuntimeError (the original exception `e`) to confirm which sizes are involved, then download the correct encoder.
Ensure image_encoder model loaded fully (no truncated/corrupt safetensors) — a partially loaded encoder can also produce shape errors.

Example fix

// before
ip_adapter = IPAdapter(model, ip_adapter_model='sdxl_ip_adapter.safetensors', image_encoder='clip-vit-large-patch14')  # wrong encoder
// after
ip_adapter = IPAdapter(model, ip_adapter_model='sdxl_ip_adapter.safetensors', image_encoder='h94/IP-Adapter/models/image_encoder')  # ViT-H, matches adapter

Defensive patterns

Strategy: try-catch

Validate before calling

from safetensors import safe_open

def check_ip_adapter_encoder(adapter_path: str, encoder_hidden_size: int) -> bool:
    with safe_open(adapter_path, framework="pt") as f:
        for k in f.keys():
            if "proj.weight" in k or "to_q.weight" in k:
                return f.get_tensor(k).shape[1] == encoder_hidden_size
    return False

Type guard

def encoder_matches_adapter(adapter, image_encoder_hidden_size: int) -> bool:
    proj = adapter._image_proj_model
    in_dim = next(proj.parameters()).shape[-1] if proj is not None else None
    return in_dim == image_encoder_hidden_size

Try / catch

try:
    embeds = ip_adapter.get_image_embeds(image)
except RuntimeError as e:
    if "incompatible with the current IP Adapter" in str(e):
        raise ModelCompatibilityError(
            "IP-Adapter checkpoint and CLIP image encoder mismatch; "
            "load the encoder listed in the adapter's README"
        ) from e
    raise

Prevention

Always pair the adapter checkpoint with the encoder named in its model card (h94/IP-Adapter ViT-H for SD1.5/most SDXL).
Keep (base_model, adapter, encoder) triples defined in one config constant instead of ad-hoc strings.
Log the chained exception (`raise ... from e`) to see the exact shape mismatch.
Pin model repo revisions so a silent upstream update cannot swap encoder dimensions.

When it happens

Trigger: Loading an IP-Adapter checkpoint whose projection layer expects a different embedding size than the supplied CLIP Vision model produces — e.g. pairing an IP-Adapter SD1.5 checkpoint (ViT-H/16, 1024-dim) with a ViT-L image encoder (768-dim), or an SDXL IP-Adapter with the wrong encoder.

Common situations: Mixing model components across SD1.5/SDXL, upgrading the IP-Adapter model version (Plus/PlusFull use different projections like Resampler vs MLPProjModel) without updating the image encoder, or copying model IDs from a tutorial for a different base model.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/f4cab04fca24d2d9. Report an issue: GitHub.