{"record":{"id":"e607a275094db1b7","repo":"unslothai/unsloth","slug":"context-does-not-support-models-loaded-with-cpu","errorCode":null,"errorMessage":"{context} does not support models loaded with CPU or disk offload. device_map='{device_map}' produced offloaded modules: {example}","messagePattern":"(.+?) does not support models loaded with CPU or disk offload\\. device_map='(.+?)' produced offloaded modules: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/utils/hardware/hardware.py","lineNumber":3888,"sourceCode":"        return {}\n    return {\n        module_name: placement\n        for module_name, placement in hf_device_map.items()\n        if placement in (\"cpu\", \"disk\")\n    }\n\n\ndef raise_if_offloaded(\n    model,\n    device_map: str,\n    context: str = \"Loading\",\n) -> None:\n    \"\"\"Raise ``ValueError`` if *model* has modules offloaded to CPU or disk.\"\"\"\n    offloaded = get_offloaded_device_map_entries(model)\n    if not offloaded:\n        return\n    example = \", \".join(f\"{name}={placement}\" for name, placement in list(offloaded.items())[:5])\n    raise ValueError(\n        f\"{context} does not support models loaded with CPU or disk offload. \"\n        f\"device_map='{device_map}' produced offloaded modules: {example}\"\n    )\n\n\ndef get_torch_device_str() -> str:\n    \"\"\"\n    Return the torch device string for the detected hardware.\n    E.g. \"cuda\", \"xpu\", or \"cpu\".\n    \"\"\"\n    device = get_device()\n    if device == DeviceType.CUDA:\n        return \"cuda\"\n    elif device == DeviceType.XPU:\n        return \"xpu\"\n    return \"cpu\"\n\n","sourceCodeStart":3870,"sourceCodeEnd":3906,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/utils/hardware/hardware.py#L3870-L3906","documentation":"raise_if_offloaded inspects a loaded model's device_map placements and raises when any modules ended up on CPU or disk (via get_offloaded_device_map_entries). The message names the calling context, the device_map string used, and up to 5 example offloaded modules with placements. It exists because many operations (e.g. LoRA/quantization paths) are incompatible with accelerate offloading.","triggerScenarios":"Loading a model with device_map='auto' (or 'balanced') when it does not fit in available VRAM, causing accelerate to place layers on CPU/disk, then calling an operation that invokes raise_if_offloaded.","commonSituations":"GPU too small for the chosen model (7B fp16 needs ~14GB+); other processes consuming VRAM; device_map='auto' on a multi-GPU node sharding to CPU; someone requesting device_map='auto' expecting pure-GPU placement.","solutions":["Free VRAM (close other jobs) or restart with a clean GPU so the model fits entirely.","Use a smaller model or quantized variant (4-bit/8-bit) so all modules fit on GPU.","Load with an explicit GPU-only device_map (e.g. device_map={'': 0}) so failure to fit surfaces at load time instead.","Add GPUs to the node / increase gpu_ids so sharding keeps everything on devices.","If CPU offload is genuinely intended, use a code path that supports it instead of the context that raises."],"exampleFix":"# before\nmodel = load(model_name, device_map=\"auto\")  # partially offloaded to CPU\nraise_if_offloaded(model, \"auto\", \"Training\")  # ValueError\n\n# after\nmodel = load(quantize_4bit(model_name), device_map={\"\": 0})  # fits fully on GPU","handlingStrategy":"validation","validationCode":"def model_fits_without_offload(model) -> bool:\n    from utils.hardware.hardware import get_offloaded_device_map_entries\n    return not get_offloaded_device_map_entries(model)\n\n# after load: assert model_fits_without_offload(model) before training","typeGuard":null,"tryCatchPattern":"try:\n    raise_if_offloaded(model, device_map=\"auto\", context=\"Training\")\nexcept ValueError as e:\n    if \"offloaded modules\" in str(e):\n        del model\n        model = load_quantized(model_name, bits=4, device_map={\"\": 0})\n        raise_if_offloaded(model, \"balanced\", \"Training\")\n    else:\n        raise","preventionTips":["Free VRAM before loading (check nvidia-smi for stray processes).","Use quantized (4/8-bit) variants for large models on small GPUs.","Load with device_map={'': 0} so misfit fails fast at load, not mid-pipeline.","Add GPUs or reduce concurrent jobs when sharding would spill to CPU."],"tags":["gpu","vram","device-map","offload","accelerate"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}