unslothai/unsloth · error · ValueError

ControlNet currently combines with plain text-to-image only,

Error message

ControlNet currently combines with plain text-to-image only, not the {workflow} workflow.

What it means

The ControlNet path is built on the plain text-to-image pipeline only. If the request selected any other workflow — inpaint, img2img, edit, reference, or upscale — the code rejects the controlnet parameter with this ValueError rather than silently ignoring it or mis-applying conditioning to a pipeline that does not accept it.

Source

Thrown at studio/backend/core/inference/diffusion.py:5420

                        decode_b64_image(x, mode = "RGB") for x in (reference_images or [])[:3]
                    ]
                elif init_image is not None:
                    workflow = "img2img"
                    pipe = self._workflow_pipe(state, state.family.img2img_pipeline_class, workflow)
                    init_pil = decode_b64_image(init_image, mode = "RGB")
                else:
                    workflow = "txt2img"

                # ControlNet (diffusers): txt2img only. Builds the family CN pipeline around resident modules.
                if controlnet is not None:
                    from core.inference import diffusion_controlnet
                    cn_id, cn_image_b64, cn_type, cn_strength, cn_gs, cn_ge = controlnet
                    # strength 0 disables CN: skip the whole path so a no-op never pays the download/VRAM.
                    if cn_strength in (None, 0, 0.0):
                        controlnet = None
                    else:
                        if workflow != "txt2img":
                            raise ValueError(
                                "ControlNet currently combines with plain text-to-image only, not "
                                f"the {workflow} workflow."
                            )
                        if not diffusion_controlnet.supports_controlnet(
                            engine = "diffusers",
                            family = state.family.name,
                            has_controlnet_pipeline = bool(
                                getattr(state.family, "controlnet_pipeline_class", None)
                            ),
                            model_kind = state.kind,
                            transformer_quant = state.transformer_quant,
                        ):
                            raise ValueError(
                                "ControlNet is not supported for this model/quantisation on the "
                                "diffusers engine (needs a bf16 or bnb-4bit load of a family with a "
                                "ControlNet pipeline; not GGUF-via-diffusers or torchao fp8/int8)."
                            )
                        # Decode + preprocess the control image FIRST so a bad image 400s before any CN download, at the OUTPUT size.

View on GitHub (pinned to 203007d190)

Solutions

  1. Drop the controlnet parameter for any conditioned workflow (img2img/inpaint/edit/reference/upscale).
  2. For ControlNet generation, send prompt + controlnet only, with no init_image/mask_image, so the workflow resolves to txt2img.
  3. Note strength 0 disables CN cleanly — but the correct fix is removing the parameter, not zeroing it.

Example fix

# before
engine.generate(prompt=p, init_image=img, controlnet=(cn_id, cn_b64, "canny", 0.8, 0.0, 1.0))
# after
engine.generate(prompt=p, controlnet=(cn_id, cn_b64, "canny", 0.8, 0.0, 1.0))  # txt2img workflow
Defensive patterns

Strategy: validation

Validate before calling

conditioned = any([init_image, mask_image]) or (upscale or 0) > 1.0
if controlnet and conditioned:
    raise UserError("ControlNet works with txt2img only; drop the CN or the conditioning image.")

Type guard

def controlnet_compatible_request(payload: dict) -> bool:
    return not (
        payload.get("controlnet")
        and (payload.get("init_image") or payload.get("mask_image") or (payload.get("upscale") or 0) > 1.0)
    )

Try / catch

try:
    out = engine.generate(**payload)
except ValueError as e:
    if "combines with plain text-to-image only" in str(e):
        payload.pop("controlnet")
        out = engine.generate(**payload)  # explicit fallback policy
    else:
        raise

Prevention

When it happens

Trigger: Sending a controlnet tuple (id, image, type, strength, gs, ge) with cn_strength not in (None, 0, 0.0) while workflow != 'txt2img' — e.g. controlnet + mask_image + init_image (inpaint), or controlnet + init_image alone (img2img).

Common situations: Adding a ControlNet depth/canny image to an img2img or inpaint request assuming diffusers composes them; UI payloads that keep the CN control attached when switching tabs to image-to-image.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/065ad9c7dc74f9e4. Report an issue: GitHub.