{"record":{"id":"065ad9c7dc74f9e4","repo":"unslothai/unsloth","slug":"controlnet-currently-combines-with-plain-text-to-i","errorCode":null,"errorMessage":"ControlNet currently combines with plain text-to-image only, not the {workflow} workflow.","messagePattern":"ControlNet currently combines with plain text-to-image only, not the (.+?) workflow\\.","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"studio/backend/core/inference/diffusion.py","lineNumber":5420,"sourceCode":"                        decode_b64_image(x, mode = \"RGB\") for x in (reference_images or [])[:3]\n                    ]\n                elif init_image is not None:\n                    workflow = \"img2img\"\n                    pipe = self._workflow_pipe(state, state.family.img2img_pipeline_class, workflow)\n                    init_pil = decode_b64_image(init_image, mode = \"RGB\")\n                else:\n                    workflow = \"txt2img\"\n\n                # ControlNet (diffusers): txt2img only. Builds the family CN pipeline around resident modules.\n                if controlnet is not None:\n                    from core.inference import diffusion_controlnet\n                    cn_id, cn_image_b64, cn_type, cn_strength, cn_gs, cn_ge = controlnet\n                    # strength 0 disables CN: skip the whole path so a no-op never pays the download/VRAM.\n                    if cn_strength in (None, 0, 0.0):\n                        controlnet = None\n                    else:\n                        if workflow != \"txt2img\":\n                            raise ValueError(\n                                \"ControlNet currently combines with plain text-to-image only, not \"\n                                f\"the {workflow} workflow.\"\n                            )\n                        if not diffusion_controlnet.supports_controlnet(\n                            engine = \"diffusers\",\n                            family = state.family.name,\n                            has_controlnet_pipeline = bool(\n                                getattr(state.family, \"controlnet_pipeline_class\", None)\n                            ),\n                            model_kind = state.kind,\n                            transformer_quant = state.transformer_quant,\n                        ):\n                            raise ValueError(\n                                \"ControlNet is not supported for this model/quantisation on the \"\n                                \"diffusers engine (needs a bf16 or bnb-4bit load of a family with a \"\n                                \"ControlNet pipeline; not GGUF-via-diffusers or torchao fp8/int8).\"\n                            )\n                        # Decode + preprocess the control image FIRST so a bad image 400s before any CN download, at the OUTPUT size.","sourceCodeStart":5402,"sourceCodeEnd":5438,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion.py#L5402-L5438","documentation":"The ControlNet path is built on the plain text-to-image pipeline only. If the request selected any other workflow — inpaint, img2img, edit, reference, or upscale — the code rejects the controlnet parameter with this ValueError rather than silently ignoring it or mis-applying conditioning to a pipeline that does not accept it.","triggerScenarios":"Sending a controlnet tuple (id, image, type, strength, gs, ge) with cn_strength not in (None, 0, 0.0) while workflow != 'txt2img' — e.g. controlnet + mask_image + init_image (inpaint), or controlnet + init_image alone (img2img).","commonSituations":"Adding a ControlNet depth/canny image to an img2img or inpaint request assuming diffusers composes them; UI payloads that keep the CN control attached when switching tabs to image-to-image.","solutions":["Drop the controlnet parameter for any conditioned workflow (img2img/inpaint/edit/reference/upscale).","For ControlNet generation, send prompt + controlnet only, with no init_image/mask_image, so the workflow resolves to txt2img.","Note strength 0 disables CN cleanly — but the correct fix is removing the parameter, not zeroing it."],"exampleFix":"# before\nengine.generate(prompt=p, init_image=img, controlnet=(cn_id, cn_b64, \"canny\", 0.8, 0.0, 1.0))\n# after\nengine.generate(prompt=p, controlnet=(cn_id, cn_b64, \"canny\", 0.8, 0.0, 1.0))  # txt2img workflow","handlingStrategy":"validation","validationCode":"conditioned = any([init_image, mask_image]) or (upscale or 0) > 1.0\nif controlnet and conditioned:\n    raise UserError(\"ControlNet works with txt2img only; drop the CN or the conditioning image.\")","typeGuard":"def controlnet_compatible_request(payload: dict) -> bool:\n    return not (\n        payload.get(\"controlnet\")\n        and (payload.get(\"init_image\") or payload.get(\"mask_image\") or (payload.get(\"upscale\") or 0) > 1.0)\n    )","tryCatchPattern":"try:\n    out = engine.generate(**payload)\nexcept ValueError as e:\n    if \"combines with plain text-to-image only\" in str(e):\n        payload.pop(\"controlnet\")\n        out = engine.generate(**payload)  # explicit fallback policy\n    else:\n        raise","preventionTips":["Treat ControlNet and image conditioning as mutually exclusive in client schemas.","Clear CN attachments when the workflow switches away from txt2img."],"tags":["diffusion","controlnet","validation","workflow"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}