unslothai/unsloth · warning · HTTPException

Cannot start training over the API while an inference reques

Error message

Cannot start training over the API while an inference request is in progress. Wait for it to finish, or start training from the Unsloth UI.

What it means

HTTP 409 raised only for API-key-authenticated training starts: Unsloth running as an inference API unloads the chat model to free VRAM for training, which would kill any in-flight inference stream. The guard checks other_inference_request_count() and background video generation before allowing the start.

Source

Thrown at studio/backend/routes/training.py:1183

                request.start_request_id,
                job_id,
            )
            if reservation == "existing":
                return _start_request_response(record)
            if reservation == "conflict":
                return _start_request_response(record)
            reserved_start_request_id = request.start_request_id

        # When Unsloth is driven as an inference API (API-key auth), refuse to start training while
        # a request is in flight: training frees VRAM by unloading the chat model, killing the
        # stream. The UI (session auth) still starts and coexists. Mixed UI+API is not special-cased.
        if via_api_key is True:
            from core.inference.llama_keepwarm import other_inference_request_count
            if (
                other_inference_request_count(current_request_counted = False) > 0
                or _background_video_generation_active()
            ):
                raise HTTPException(
                    status_code = 409,
                    detail = (
                        "Cannot start training over the API while an inference request is in "
                        "progress. Wait for it to finish, or start training from the Unsloth UI."
                    ),
                )

        # No in-process ensure_transformers_version(): worker.py activates it before ML imports.

        # A consented latest-transformers install stage-and-swaps .venv_t5_latest mid-spawn.
        from utils.transformers_latest import is_install_in_progress

        if is_install_in_progress():
            raise HTTPException(
                status_code = 409,
                detail = ("A transformers installation is in progress. Retry when it completes."),
            )

View on GitHub (pinned to 203007d190)

Solutions

Wait for in-flight inference requests (and background video generation) to finish, then retry the training start
Start training from the Unsloth UI with session auth instead - the UI path coexists with inference and is not blocked
Drain and queue incoming inference requests before calling the API-key training start

Example fix

# before
client.post("/training/start", payload)  # 409 while a stream is open

# after
# close/finish active chat completions first, then
resp = client.post("/training/start", payload)
if resp.status_code == 409:
    time.sleep(5); resp = client.post("/training/start", payload)

Defensive patterns

Strategy: retry

Validate before calling

# client-side: ensure no open inference streams before API-key training start
assert not active_chat_streams(), "drain inference requests first"
assert not video_generation_in_flight(), "wait for background video clip"

Try / catch

resp = client.post("/training/start", payload)
if resp.status_code == 409 and "inference request is in progress" in resp.text:
    wait_for_inference_drain()          # poll your own stream bookkeeping
    resp = client.post("/training/start", payload)

Prevention

Close all streaming chat completions and wait out background video generation before API-key training starts
Gate training starts behind an inference-idle check in your orchestrator
For UI-driven workflows use session auth, which is not subject to this guard

When it happens

Trigger: POST /training/start with API-key auth while another inference request is streaming, or while a background video clip is generating on the video backend's worker thread.

Common situations: Serving an OpenAI-compatible endpoint and kicking off fine-tuning without draining requests; a long chat completion still streaming when the training API call lands; video generation running in background.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/560bb71ec50b8cda. Report an issue: GitHub.