{"record":{"id":"a807c1fa64bc84c7","repo":"unslothai/unsloth","slug":"no-model-loaded","errorCode":null,"errorMessage":"No model loaded","messagePattern":"No model loaded","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/mlx_inference.py","lineNumber":1438,"sourceCode":"        image = None,\n        temperature = 0.7,\n        top_p = 0.9,\n        top_k = 40,\n        min_p = 0.0,\n        max_new_tokens = 256,\n        repetition_penalty = 1.0,\n        cancel_event = None,\n        # Reasoning / tool kwargs, rendered via apply_chat_template_for_generation (transformers parity).\n        tools = None,\n        enable_thinking = None,\n        reasoning_effort = None,\n        preserve_thinking = None,\n        continue_final_message = False,\n        presence_penalty = 0.0,\n        _adapter_state = None,\n    ) -> Generator[str, None, None]:\n        if self._model is None:\n            raise RuntimeError(\"No model loaded\")\n\n        # Reset so a failed run cannot surface stale stats.\n        self.last_generation_stats = None\n\n        full_messages = []\n        if system_prompt:\n            full_messages.append({\"role\": \"system\", \"content\": system_prompt})\n        full_messages.extend(messages)\n\n        # Inject image into the last user message for VLM\n        if self._is_vlm and image is not None:\n            for msg in reversed(full_messages):\n                if msg.get(\"role\") == \"user\":\n                    content = msg.get(\"content\", \"\")\n                    if isinstance(content, str):\n                        msg[\"content\"] = [\n                            {\"type\": \"image\"},\n                            {\"type\": \"text\", \"text\": content},","sourceCodeStart":1420,"sourceCodeEnd":1456,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/mlx_inference.py#L1420-L1456","documentation":"Straightforward guard at the top of the MLX generation entry point: if self._model is None the backend has no loaded model, so streaming cannot proceed and RuntimeError('No model loaded') is raised immediately. This happens when generation is invoked before a successful load() or after the model was unloaded/crashed.","triggerScenarios":"Calling stream/generate on a fresh MLXInferenceBackend instance without load(); calling after unload(); calling after a load that raised (the error path left _model None).","commonSituations":"API route hit before the model finished loading (race at service startup); load failed with an earlier error and the caller ignored it; client reconnecting after a model-swap sequence.","solutions":["Load a model first and confirm it succeeded (check active_model_name / load result) before issuing generation requests.","If loading previously failed, address that root-cause error — 'No model loaded' is only the downstream symptom.","Gate the API route: return 503 until the backend reports a loaded model."],"exampleFix":"# before\nbackend = MLXInferenceBackend(...)\nout = backend.stream_response(messages)  # RuntimeError: No model loaded\n\n# after\nbackend.load('mlx-community/Llama-3.1-8B-Instruct-4bit')\nout = backend.stream_response(messages)","handlingStrategy":"type-guard","validationCode":"if backend._model is None or not backend.active_model_name:\n    raise ServiceUnavailable('model not loaded; call load() first')","typeGuard":"def has_loaded_model(backend) -> bool:\n    return getattr(backend, '_model', None) is not None","tryCatchPattern":"try:\n    for chunk in backend.stream_response(messages):\n        yield chunk\nexcept RuntimeError as e:\n    if str(e) == 'No model loaded':\n        yield error_event('model_not_loaded', retry_after_load=True)\n    else:\n        raise","preventionTips":["Gate inference routes on a loaded-model health check; return 503 until load completes.","Never ignore load() failures — this error is the downstream symptom.","Track load state explicitly in the orchestrator and reject generation during model swaps."],"tags":["mlx","lifecycle","model-loading","inference"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}