unslothai/unsloth · warning · HTTPException

Cannot start diffusion (Images) training over the API while

Error message

Cannot start diffusion (Images) training over the API while an inference request is in progress. Wait for it to finish, or start training from the Studio UI.

What it means

HTTP 409 on the diffusion training start endpoint when called with API-key authentication while other inference requests (chat streams) or background video generation are in progress. Reason: _free_gpu_for_diffusion_training() unloads the chat backends to make room for the SDXL/DiT pipeline, which would kill the in-flight inference streams. The same operation from the Studio UI (session auth) is allowed.

Source

Thrown at studio/backend/routes/training.py:2705

@router.post("/diffusion/start", response_model = DiffusionTrainingStartResponse)
async def start_diffusion_training(
    body: DiffusionTrainingStartRequest,
    current_subject: str = Depends(get_current_subject),
    via_api_key: bool = Depends(authenticated_via_api_key),
):
    """Start an SDXL LoRA training job from an image + caption dataset."""
    from core.training.diffusion_training_service import get_diffusion_training_service

    # Under API-key auth, refuse to start training while a request is in flight: _free_gpu_for_diffusion_training() below unloads the chat backends, killing the stream. Mirrors start_training.
    if via_api_key is True:
        from core.inference.llama_keepwarm import other_inference_request_count
        if (
            other_inference_request_count(current_request_counted = False) > 0
            or _background_video_generation_active()
        ):
            raise HTTPException(
                status_code = 409,
                detail = (
                    "Cannot start diffusion (Images) training over the API while an inference "
                    "request is in progress. Wait for it to finish, or start training from the "
                    "Studio UI."
                ),
            )

    # Interlock: refuse while an LLM training run holds the GPU (symmetric with the diffusion check in start_training), so the two trainers never contend for VRAM.
    try:
        if get_training_backend().is_training_active():
            raise HTTPException(
                status_code = 409,
                detail = (
                    "An LLM training job is already running. "
                    "Stop it before starting diffusion (Images) training."
                ),
            )

View on GitHub (pinned to 203007d190)

Solutions

1. Wait for in-flight inference requests and background video generation to finish, then retry.
2. Or start the training from the Studio UI, which does not run under this restriction.
3. In automation, gate the training call on an inference-idle check (no active streams/video jobs) first.

if (usingApiKey) {
  const inferenceBusy = (await get('/inference/active-count')).count > 0
    || (await get('/video/background-status')).active
  if (inferenceBusy) throw new Error('Wait for inference/video to finish, or start training from the Studio UI')
}

try { await startDiffusionTraining(payload) } catch (e) { if (e.status === 409 && /inference request is in progress/.test(e.detail)) { await waitForInferenceIdle(); return startDiffusionTraining(payload) } throw e }

When it happens

Trigger: POST /training/diffusion/start (or equivalent) with an API key while other_inference_request_count()>0 or a background video generation job is active.

Common situations: Automations/bots starting image training while the same instance serves chat requests; a background video render still running when the API training call fires.

Related errors

• An LLM training job is already running. Stop it before start
• Cannot start training over the API while an inference reques
• transformer_quant='{requested}' could not be used: {reason}.
• The current inference worker did not exit and still holds GP
• {str(exc)}

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/c54f6847d981af40. Report an issue: GitHub.