{"record":{"id":"bfcc7a4cefcc0722","repo":"unslothai/unsloth","slug":"a-transformers-installation-is-replacing-the-lates-bfcc7a","errorCode":null,"errorMessage":"A transformers installation is replacing the latest sidecar; retry when it completes.","messagePattern":"A transformers installation is replacing the latest sidecar; retry when it completes\\.","errorType":"http","errorClass":"SidecarSwapInProgress","httpStatus":409,"severity":"warning","filePath":"studio/backend/routes/training.py","lineNumber":1685,"sourceCode":"                    backend,\n                    reserved_start_request_id,\n                    str(exc),\n                )\n                raise\n\n        try:\n            start_task = asyncio.create_task(asyncio.to_thread(_run_backend_start))\n            success = await asyncio.shield(start_task)\n        except _DiffusionStartInFlight as exc:\n            return TrainingJobResponse(\n                job_id = \"\",\n                status = \"error\",\n                message = str(exc),\n                error = \"Diffusion training already active\",\n            )\n        except SidecarSwapInProgress as exc:\n            # Expected loss of the race against a sidecar install: a retryable 409, not an internal error.\n            raise HTTPException(status_code = 409, detail = str(exc))\n        except ExactResumeResourcesUnavailable as exc:\n            raise HTTPException(status_code = 409, detail = str(exc))\n\n        if not success:\n            progress_error = backend.trainer.training_progress.error\n            failure_message = progress_error or \"Failed to start training subprocess\"\n            return TrainingJobResponse(\n                job_id = backend.current_job_id or \"\",\n                status = \"error\",\n                message = failure_message,\n                error = progress_error or \"subprocess_start_failed\",\n            )\n\n        return TrainingJobResponse(\n            job_id = job_id,\n            status = \"queued\",\n            message = \"Training job queued and starting in subprocess\",\n            error = None,","sourceCodeStart":1667,"sourceCodeEnd":1703,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/routes/training.py#L1667-L1703","documentation":"HTTP 409 from SidecarSwapInProgress: the training start raced against an in-progress transformers sidecar install/swap and lost. The comment in the route explicitly calls it 'a retryable 409, not an internal error' — the fix is to wait for the sidecar installation to complete and retry the same request.","triggerScenarios":"POST /training/start (or the resume path) while a 'latest transformers' sidecar installation is concurrently replacing the transformers install; the backend's start sequence detects the swap mid-flight and aborts cleanly.","commonSituations":"A user (or another tab/automation) triggered a transformers upgrade on the settings page and then immediately started training; CI starting training right after requesting a runtime upgrade.","solutions":["Wait for the transformers sidecar installation to finish, then retry the identical start request.","Check the installation status endpoint/UI before retrying so you are not racing again.","Serialize runtime swaps and training starts in your automation (never trigger an install between a start attempt and its retry)."],"exampleFix":"// before: fire-and-forget start\nawait post('/training/start', payload)\n// after: honor the retryable 409\nresp = await post('/training/start', payload)\nif resp.status === 409 && /replacing the latest sidecar/.test(resp.detail) {\n  await waitForSidecarIdle()\n  resp = await post('/training/start', payload)\n}","handlingStrategy":"retry","validationCode":"const install = await get('/settings/sidecar/status')\nif (install.in_progress) await waitForSidecarIdle() // poll until install finishes\nawait post('/training/start', payload)","typeGuard":null,"tryCatchPattern":"async function startWithRetry(payload, attempts = 5) {\n  for (;;) {\n    try { return await post('/training/start', payload) }\n    catch (e) {\n      if (e.status === 409 && /replacing the latest sidecar/.test(e.detail) && --attempts > 0) {\n        await sleep(10_000); continue\n      }\n      throw e\n    }\n  }\n}","preventionTips":["Serialize sidecar installs and training starts — never interleave them in automation.","Check sidecar/installation status before issuing start requests.","Recognize this exact 409 as retryable by message pattern, and back off several seconds between retries."],"tags":["sidecar","race-condition","transformers","retryable","http-409"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}