{"record":{"id":"f4cab04fca24d2d9","repo":"invoke-ai/InvokeAI","slug":"selected-clip-vision-model-is-incompatible-with-th","errorCode":null,"errorMessage":"Selected CLIP Vision Model is incompatible with the current IP Adapter","messagePattern":"Selected CLIP Vision Model is incompatible with the current IP Adapter","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/ip_adapter/ip_adapter.py","lineNumber":156,"sourceCode":"        from invokeai.backend.model_manager.load.model_util import calc_module_size\n\n        return calc_module_size(self._image_proj_model) + calc_module_size(self.attn_weights)\n\n    def _init_image_proj_model(\n        self, state_dict: dict[str, torch.Tensor]\n    ) -> Union[ImageProjModel, Resampler, MLPProjModel]:\n        return ImageProjModel.from_state_dict(state_dict, self._num_tokens).to(self.device, dtype=self.dtype)\n\n    @torch.inference_mode()\n    def get_image_embeds(self, pil_image: List[Image.Image], image_encoder: CLIPVisionModelWithProjection):\n        clip_image = self._clip_image_processor(images=pil_image, return_tensors=\"pt\").pixel_values\n        clip_image_embeds = image_encoder(clip_image.to(self.device, dtype=self.dtype)).image_embeds\n        try:\n            image_prompt_embeds = self._image_proj_model(clip_image_embeds)\n            uncond_image_prompt_embeds = self._image_proj_model(torch.zeros_like(clip_image_embeds))\n            return image_prompt_embeds, uncond_image_prompt_embeds\n        except RuntimeError as e:\n            raise RuntimeError(\"Selected CLIP Vision Model is incompatible with the current IP Adapter\") from e\n\n\nclass IPAdapterPlus(IPAdapter):\n    \"\"\"IP-Adapter with fine-grained features\"\"\"\n\n    def _init_image_proj_model(self, state_dict: dict[str, torch.Tensor]) -> Union[Resampler, MLPProjModel]:\n        return Resampler.from_state_dict(\n            state_dict=state_dict,\n            depth=4,\n            dim_head=64,\n            heads=12,\n            num_queries=self._num_tokens,\n            ff_mult=4,\n        ).to(self.device, dtype=self.dtype)\n\n    @torch.inference_mode()\n    def get_image_embeds(self, pil_image: List[Image.Image], image_encoder: CLIPVisionModelWithProjection):\n        clip_image = self._clip_image_processor(images=pil_image, return_tensors=\"pt\").pixel_values","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/ip_adapter/ip_adapter.py#L138-L174","documentation":"IPAdapter.get_image_embeds runs the CLIP vision encoder output through the adapter's image projection model (_image_proj_model). If that forward pass raises a RuntimeError (typically a torch matmul/shape mismatch), the library re-raises with this message because it means the image encoder's embedding dimension does not match what the IP-Adapter weights were trained for.","triggerScenarios":"Loading an IP-Adapter checkpoint whose projection layer expects a different embedding size than the supplied CLIP Vision model produces — e.g. pairing an IP-Adapter SD1.5 checkpoint (ViT-H/16, 1024-dim) with a ViT-L image encoder (768-dim), or an SDXL IP-Adapter with the wrong encoder.","commonSituations":"Mixing model components across SD1.5/SDXL, upgrading the IP-Adapter model version (Plus/PlusFull use different projections like Resampler vs MLPProjModel) without updating the image encoder, or copying model IDs from a tutorial for a different base model.","solutions":["Use the image encoder paired with the IP-Adapter checkpoint: SD1.5 IP-Adapters require 'h94/IP-Adapter' ViT-H (models/image_encoder/model.safetensors); SDXL IP-Adapter uses ViT-H as well, while some SDXL adapters use ViT-bigG — check the checkpoint's README.","Verify the IP-Adapter variant matches (IPAdapter vs IPAdapterPlus vs IPAdapterFull) — each initializes a different projection model.","Check the embedding dimension mismatch in the chained RuntimeError (the original exception `e`) to confirm which sizes are involved, then download the correct encoder.","Ensure image_encoder model loaded fully (no truncated/corrupt safetensors) — a partially loaded encoder can also produce shape errors."],"exampleFix":"// before\nip_adapter = IPAdapter(model, ip_adapter_model='sdxl_ip_adapter.safetensors', image_encoder='clip-vit-large-patch14')  # wrong encoder\n// after\nip_adapter = IPAdapter(model, ip_adapter_model='sdxl_ip_adapter.safetensors', image_encoder='h94/IP-Adapter/models/image_encoder')  # ViT-H, matches adapter","handlingStrategy":"try-catch","validationCode":"from safetensors import safe_open\n\ndef check_ip_adapter_encoder(adapter_path: str, encoder_hidden_size: int) -> bool:\n    with safe_open(adapter_path, framework=\"pt\") as f:\n        for k in f.keys():\n            if \"proj.weight\" in k or \"to_q.weight\" in k:\n                return f.get_tensor(k).shape[1] == encoder_hidden_size\n    return False","typeGuard":"def encoder_matches_adapter(adapter, image_encoder_hidden_size: int) -> bool:\n    proj = adapter._image_proj_model\n    in_dim = next(proj.parameters()).shape[-1] if proj is not None else None\n    return in_dim == image_encoder_hidden_size","tryCatchPattern":"try:\n    embeds = ip_adapter.get_image_embeds(image)\nexcept RuntimeError as e:\n    if \"incompatible with the current IP Adapter\" in str(e):\n        raise ModelCompatibilityError(\n            \"IP-Adapter checkpoint and CLIP image encoder mismatch; \"\n            \"load the encoder listed in the adapter's README\"\n        ) from e\n    raise","preventionTips":["Always pair the adapter checkpoint with the encoder named in its model card (h94/IP-Adapter ViT-H for SD1.5/most SDXL).","Keep (base_model, adapter, encoder) triples defined in one config constant instead of ad-hoc strings.","Log the chained exception (`raise ... from e`) to see the exact shape mismatch.","Pin model repo revisions so a silent upstream update cannot swap encoder dimensions."],"tags":["pytorch","ip-adapter","clip","model-compatibility","shape-mismatch"],"backgroundTag":"model-incompatible-weights","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}