{"record":{"id":"f28028d44bfae808","repo":"unslothai/unsloth","slug":"unsloth-mlx-inference-requires-unsloth-zoo-with-t","errorCode":null,"errorMessage":"Unsloth: MLX inference requires unsloth-zoo with the MLX modules (unsloth_zoo.mlx.loader). Reinstall via install.sh on Apple Silicon.","messagePattern":"Unsloth: MLX inference requires unsloth-zoo with the MLX modules \\(unsloth_zoo\\.mlx\\.loader\\)\\. Reinstall via install\\.sh on Apple Silicon\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/mlx_inference.py","lineNumber":1186,"sourceCode":"            distributed_size,\n            parallel_mode,\n        )\n        if is_distributed and parallel_mode not in (\"pipeline\", \"tensor\"):\n            raise ValueError(\n                \"Unsloth: distributed MLX inference requires parallel_mode='pipeline' \"\n                \"or parallel_mode='tensor'.\"\n            )\n        if is_distributed and is_lora:\n            raise ValueError(\n                \"Unsloth: distributed MLX inference for LoRA adapter repos \"\n                \"is not supported yet. Merge/export the adapter into an MLX model \"\n                \"before distributed inference.\"\n            )\n\n        try:\n            from unsloth_zoo.mlx.loader import FastMLXModel\n        except ImportError as e:\n            raise ImportError(\n                \"Unsloth: MLX inference requires unsloth-zoo with the MLX modules \"\n                \"(unsloth_zoo.mlx.loader). Reinstall via install.sh on Apple Silicon.\"\n            ) from e\n\n        load_kwargs = {\n            \"max_seq_length\": max_seq_length,\n            \"dtype\": dtype,\n            \"load_in_4bit\": load_in_4bit,\n            \"token\": hf_token,\n            \"trust_remote_code\": trust_remote_code,\n            \"text_only\": False if is_vision else True,\n        }\n        if is_distributed:\n            if parallel_mode == \"pipeline\":\n                load_kwargs[\"pipeline_group\"] = distributed_group\n            else:\n                load_kwargs[\"tensor_group\"] = distributed_group\n","sourceCodeStart":1168,"sourceCodeEnd":1204,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/mlx_inference.py#L1168-L1204","documentation":"ImportError raised when `from unsloth_zoo.mlx.loader import FastMLXModel` fails inside the MLX backend's load path. The MLX inference stack lives in unsloth-zoo's MLX modules, which are only installed by install.sh on Apple Silicon; a Linux install, a pip install of unsloth-zoo without the MLX extras, or a broken environment makes the import fail. The original ImportError is chained as the cause.","triggerScenarios":"Loading any model through the MLX backend on a machine where unsloth_zoo.mlx.loader is absent: plain pip install unsloth-zoo (no MLX modules), Linux/x86 hosts, or a partially failed install.sh run.","commonSituations":"Running the studio backend on Linux where it fell back to the MLX orchestrator; mixed conda/venv environments where unsloth-zoo was installed without MLX deps; Apple Silicon box where install.sh was interrupted or the venv was recreated by hand.","solutions":["Run the project's install.sh on an Apple Silicon macOS machine — it installs unsloth-zoo with the MLX modules.","Verify the module resolves: python -c \"from unsloth_zoo.mlx.loader import FastMLXModel\" and fix the environment it fails in.","On non-Apple-Silicon hosts, use a supported backend (llama-server/GGUF or transformers) instead of the MLX orchestrator."],"exampleFix":"# before (linux / incomplete env)\nbackend.load('mlx-community/Llama-3.1-8B-Instruct-4bit')  # ImportError\n\n# after\n# on Apple Silicon macOS:\n# ./install.sh  (installs unsloth-zoo with MLX modules)\nbackend.load('mlx-community/Llama-3.1-8B-Instruct-4bit')","handlingStrategy":"validation","validationCode":"def mlx_backend_available() -> bool:\n    try:\n        import unsloth_zoo.mlx.loader  # noqa: F401\n        return True\n    except ImportError:\n        return False\n\nif not mlx_backend_available():\n    disable_route('mlx')  # or fall back to transformers/llama-server","typeGuard":null,"tryCatchPattern":"try:\n    backend.load(model_name)\nexcept ImportError as e:\n    if 'unsloth_zoo.mlx.loader' in str(e):\n        raise RuntimeError('MLX backend unavailable on this host; use llama-server backend') from e\n    raise","preventionTips":["Feature-probe for unsloth_zoo.mlx.loader at service startup and hide/disable the MLX route on hosts without it.","Provision Apple Silicon environments exclusively through install.sh so MLX modules are always present.","Fail fast in deployment checks rather than at first model load."],"tags":["mlx","unsloth-zoo","installation","apple-silicon","import-error"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}