{"record":{"id":"51838a81d0fb6d7a","repo":"unslothai/unsloth","slug":"text-encoder-quant-requested-could-not-be-used-51838a","errorCode":null,"errorMessage":"text_encoder_quant='{requested}' could not be used: {reason}. Leave it unset to keep the dense bf16 encoder.","messagePattern":"text_encoder_quant='(.+?)' could not be used: (.+?)\\. Leave it unset to keep the dense bf16 encoder\\.","errorType":"http","errorClass":"RuntimeError","httpStatus":409,"severity":"error","filePath":"studio/backend/core/inference/video.py","lineNumber":344,"sourceCode":"        # Same as the image gate: the casters import torchao only after the pipeline is built.\n        te_reason = (\n            \"torchao is not importable on this install, and these encoder modes are torchao \"\n            \"quantisations\"\n        )\n    elif te_effective is not None and not te_quant_supported(target, te_effective):\n        te_reason = (\n            \"this device does not have the tensor cores that backend needs (a CUDA GPU in \"\n            \"bf16, plus fp8 / int8 / NVFP4 support depending on the mode)\"\n        )\n    elif te_quant_needs_resident_weights(te_effective) and forces_offload:\n        # Same fence on the encoder: the loader reports those modes unsupported once offload is\n        # active, and by then the resident model is gone. Layerwise fp8 is a dtype cast.\n        te_reason = (\n            f\"'{normalize_memory_mode(memory_mode)}' memory places the text encoder under CPU \"\n            \"offload, and torchao quantised tensors cannot be moved by the offload hooks\"\n        )\n    if te_reason is not None:\n        raise RuntimeError(\n            precision_refusal_message(\n                \"text_encoder_quant\",\n                te_mode,\n                te_reason,\n                off_label = \"leave it unset to keep the dense bf16 encoder\",\n                auto_available = False,\n            )\n        )\n\n\ndef _is_trusted_video_repo(repo_id: str) -> bool:\n    \"\"\"Whether a NON-GGUF load may deserialise this repo (see the image twin).\"\"\"\n    try:\n        if Path(repo_id).expanduser().exists():\n            return True\n    except OSError:\n        pass\n    rid = repo_id.strip().lower()","sourceCodeStart":326,"sourceCodeEnd":362,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/video.py#L326-L362","documentation":"The text_encoder_quant gate refuses a pinned encoder quantization in two situations: the device lacks the tensor cores the mode needs (no CUDA GPU with the required bf16/fp8/int8/NVFP4 support), or the memory mode forces CPU offload while the mode requires resident weights (torchao tensors cannot be moved by the offload hooks; layerwise fp8 is exempt as a plain dtype cast). Auto is not offered here — the fallback is simply leaving it unset.","triggerScenarios":"Passing text_encoder_quant with a non-NVIDIA or older GPU (te_reason set by the tensor-core branch), or combining an offload-forcing memory_mode with a quant mode flagged by te_quant_needs_resident_weights (te_effective).","commonSituations":"Enabling encoder quant on Apple Silicon or AMD; setting an aggressive memory_mode ('low' style offload) together with int8/NVFP4 encoder quant; configs copied from a 24 GB+ GPU host to a small-VRAM machine where offload is auto-selected.","solutions":["Leave text_encoder_quant unset to keep the dense bf16 encoder — the always-safe option.","If you need encoder quant, switch memory_mode so the text encoder stays resident (no CPU offload).","On non-CUDA hosts, drop encoder quant entirely; only CUDA GPUs with the right tensor cores support these modes."],"exampleFix":"# before\nload_video_model(repo_id=..., text_encoder_quant='int8', memory_mode='low')\n\n# after\nload_video_model(repo_id=..., text_encoder_quant=None, memory_mode='low')","handlingStrategy":"fallback","validationCode":"# before requesting: encoder quant needs CUDA tensor cores AND resident weights\nimport torch\ndef encoder_quant_safe(requested, memory_mode_forces_offload):\n    if requested in (None, ''):\n        return True\n    if not (torch.cuda.is_available()):\n        return False\n    return not memory_mode_forces_offload  # layerwise fp8 excepted server-side","typeGuard":null,"tryCatchPattern":"try:\n    load_video_model(repo_id=r, text_encoder_quant=te, memory_mode=mm)\nexcept RuntimeError as e:\n    if 'text_encoder_quant' in str(e):\n        load_video_model(repo_id=r, text_encoder_quant=None, memory_mode=mm)\n    else:\n        raise","preventionTips":["Leave text_encoder_quant unset unless you have verified the host supports it.","Never combine encoder quant with an offloading memory mode (except layerwise fp8).","Test pinned quant configs per host class in CI, not just on the dev GPU."],"tags":["video","quantization","offload","hardware-support"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}