{"record":{"id":"36a89f5f195e0fcc","repo":"immich-app/immich","slug":"failed-to-load-model-model-model-name","errorCode":null,"errorMessage":"Failed to load model '{model.model_name}'","messagePattern":"Failed to load model '(.+?)'","errorType":"http","errorClass":"HTTPException","httpStatus":500,"severity":"error","filePath":"machine-learning/immich_ml/main.py","lineNumber":228,"sourceCode":"        response[\"imageHeight\"], response[\"imageWidth\"] = payload.height, payload.width\n\n    return response\n\n\nasync def run(func: Callable[..., T], *args: Any, **kwargs: Any) -> T:\n    if thread_pool is None:\n        return func(*args, **kwargs)\n    partial_func = partial(func, *args, **kwargs)\n    return await asyncio.get_running_loop().run_in_executor(thread_pool, partial_func)\n\n\nasync def load(model: InferenceModel) -> InferenceModel:\n    if model.loaded:\n        return model\n\n    def _load(model: InferenceModel) -> InferenceModel:\n        if model.load_attempts > 1:\n            raise HTTPException(500, f\"Failed to load model '{model.model_name}'\")\n        with lock:\n            try:\n                model.load()\n            except FileNotFoundError as e:\n                if model.model_format == ModelFormat.ONNX:\n                    raise e\n                log.warning(\n                    f\"{model.model_format.upper()} is available, but model '{model.model_name}' does not support it.\",\n                    exc_info=e,\n                )\n                model.model_format = ModelFormat.ONNX\n                model.load()\n        return model\n\n    try:\n        return await run(_load, model)\n    except (OSError, InvalidProtobuf, BadZipFile, NoSuchFile):\n        log.warning(f\"Failed to load {model.model_type.replace('_', ' ')} model '{model.model_name}'. Clearing cache.\")","sourceCodeStart":210,"sourceCodeEnd":246,"githubUrl":"https://github.com/immich-app/immich/blob/199723261c6ffa897fec8ccdaea6359e39c37cc3/machine-learning/immich_ml/main.py#L210-L246","documentation":"Raised as HTTPException(500) by the load() helper in main.py when a model fails to load more than once (model.load_attempts > 1). The first failure triggers a fallback: if the model is not ONNX format, it retries with ONNX; the second consecutive failure is treated as a hard error.","triggerScenarios":"A model's weights/session cannot be created — missing/corrupt model files, incompatible ONNX runtime, out-of-memory, unsupported model format — and the ONNX fallback also fails.","commonSituations":"Model files not fully downloaded or cache corrupted; ONNX runtime version mismatch with the model; insufficient RAM/VRAM; ARM/CPU without the right execution provider; mismatched Immich ML and model versions.","solutions":["Clear the ML model cache so models re-download fresh.","Check ML container logs for the underlying error (OOM, FileNotFoundError, provider errors).","Ensure enough memory is available; for GPU builds, verify CUDA/execution-provider compatibility.","Pin the CLIP/recognition models to ones bundled/known-good for your Immich version."],"exampleFix":"# before — model cache may be corrupt\n# (no action)\n\n# after\ndocker exec immich-machine-learning rm -rf /cache/semanticsearch /cache/facial-recognition\n# then restart the ML container and retry the job","handlingStrategy":"fallback","validationCode":"# pre-flight: ensure cache dir is writable and model files exist\nfrom pathlib import Path\nif not Path(model.cache_dir).exists(): log.warning('cache missing, will download')","typeGuard":"def can_load(model) -> bool:\n    try:\n        return Path(model.cache_dir).is_dir()\n    except Exception:\n        return False","tryCatchPattern":"try:\n    await load(model)\nexcept HTTPException as e:\n    if e.status_code == 500 and 'Failed to load model' in (e.detail or ''):\n        clear_and_reload(model)\n    raise","preventionTips":["Provide enough memory for the configured models.","Keep model cache on reliable storage; clear and re-download on corruption.","Pin known-good model versions for your Immich release."],"tags":["machine-learning","model-loading","onnx","cache","oom","fastapi"],"backgroundTag":null,"analyzedSha":"199723261c6ffa897fec8ccdaea6359e39c37cc3","analyzedAt":"2026-08-12T04:54:27.085Z","schemaVersion":2},"datasetVersion":"2026-08-12T13:17:24.610Z"}