{"record":{"id":"38f5069d12ceb005","repo":"unslothai/unsloth","slug":"transformer-quant-requested-could-not-be-used","errorCode":null,"errorMessage":"transformer_quant='{requested}' could not be used: {reason}. Choose Auto to let the backend pick the fastest precision this host can run, or Off to run the checkpoint as-is.","messagePattern":"transformer_quant='(.+?)' could not be used: (.+?)\\. Choose Auto to let the backend pick the fastest precision this host can run, or Off to run the checkpoint as-is\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/diffusion.py","lineNumber":1224,"sourceCode":"            elif (\n                select_transformer_quant_scheme(\n                    target,\n                    pinned,\n                    family = getattr(fam, \"name\", None),\n                    # This gate runs BEFORE the arbiter evicts the resident model, so a smoke probe\n                    # that runs out of VRAM here has not shown the scheme unusable, only that the\n                    # GPU is still full. Refusing on that would reject a load the eviction was\n                    # about to make room for.\n                    unproven_ok = True,\n                )\n                is None\n            ):\n                # An explicit scheme is never swapped for another, so a None means the family's\n                # measured deny list, a torchao that cannot run here, or a GPU without the kernels.\n                # They read the same to the selector and need different fixes from the user.\n                reason = explain_unusable_scheme(getattr(fam, \"name\", None), pinned)\n            if reason is not None:\n                raise RuntimeError(\n                    precision_refusal_message(\n                        \"transformer_quant\",\n                        pinned,\n                        reason,\n                        off_label = \"Off to run the checkpoint as-is\",\n                    )\n                )\n        # The mode the loader will ACTUALLY attempt, not the raw request: an explicit int8\n        # on a family with no keep-bf16 schedule is rewritten to layerwise fp8 before\n        # support is consulted, and that path needs no torchao. Refusing on the raw int8\n        # rejected loads the runtime would run and report as fell_back -- Windows ROCm,\n        # where the torchao stub kills int8 while fp8 still works.\n        te_effective = effective_te_quant(te_mode, getattr(fam, \"name\", None))\n        te_reason = None\n        if te_quant_needs_resident_weights(te_effective) and not torchao_quantize_importable():\n            # The casters import torchao only after the pipeline is downloaded and built, so a\n            # broken or absent install failed through load-progress instead of this 409.\n            te_reason = (","sourceCodeStart":1206,"sourceCodeEnd":1242,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion.py#L1206-L1242","documentation":"Raised by assert_precision_available when a user-requested transformer_quant scheme (an explicit precision like int8/fp8/nvfp4, never 'Auto') was measured or reasoned to be unusable on this host: the family's measured deny list, a torchao version that cannot run here, or a GPU lacking the required kernels. Because an explicit scheme is never silently swapped for another, a None from the quant planner is a hard refusal, formatted via precision_refusal_message with the reason from explain_unusable_scheme.","triggerScenarios":"Passing transformer_quant='<scheme>' (e.g. 'int8', 'nvfp4') to a diffusion load when the host GPU lacks the kernels for it, the running torchao build does not support it, or the family has been measured to fail with it. Note an explicit int8 on a family with no keep-bf16 schedule is rewritten to layerwise fp8 before support is consulted, so that path refuses only if fp8 also cannot run.","commonSituations":"Requesting nvfp4 on a non-Blackwell GPU; requesting int8 torchao quantization on Windows ROCm where the torchao stub kills int8; a torchao version mismatch after an environment upgrade; a family whose deny list was populated by a previous failed measurement on this same host.","solutions":["Retrigger the load with transformer_quant unset or 'Auto' - the backend then picks the fastest precision this host can actually run.","Set transformer_quant to 'Off' (or the equivalent off value) to load the checkpoint in its native precision.","If the explicit scheme is required, fix the host: upgrade/downgrade torchao to a version with the needed kernels, or run on a GPU generation that supports the scheme (e.g. nvfp4 needs Blackwell).","Check the reason string in the message - it distinguishes a family deny-list entry (pick another model) from a host kernel gap (fix the environment)."],"exampleFix":"# before: explicit scheme the host cannot run\nmanager.load(repo_id=\"unsloth/FLUX.1-dev\", transformer_quant=\"nvfp4\")\n# -> RuntimeError: transformer_quant='nvfp4' could not be used: ...\n\n# after: let the backend choose\nmanager.load(repo_id=\"unsloth/FLUX.1-dev\", transformer_quant=\"auto\")","handlingStrategy":"validation","validationCode":"# The backend exposes the same fence it uses internally:\nfam = manager.detect_family_for_pick(repo_id)  # or detect via the public API surface\nmanager.assert_precision_available(\n    fam, model_kind=resolve_model_kind(gguf_filename, model_kind),\n    transformer_quant=\"nvfp4\", text_encoder_quant=None, gpu_ordinal=0,\n)","typeGuard":null,"tryCatchPattern":"try:\n    manager.load(repo_id=repo, transformer_quant=requested)\nexcept RuntimeError as e:\n    if str(e).startswith(\"transformer_quant=\"):\n        log.warning(\"Host cannot run %s; retrying with auto\", requested)\n        manager.load(repo_id=repo, transformer_quant=\"auto\")\n    else:\n        raise","preventionTips":["Default to 'auto' for transformer_quant and only pin a scheme after it has succeeded once on the same GPU + torchao build.","Treat a scheme that failed as host-specific: record the (gpu, torchao version, scheme) tuple so the UI hides known-bad options.","Upgrade torchao and GPU drivers together; a kernel gap usually traces to one of the two.","nvfp4 needs Blackwell-class hardware; do not offer it in the picker on older GPUs."],"tags":["quantization","torchao","gpu","precision","diffusion","host-capability"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}