{"record":{"id":"f2b4c825e547d2d0","repo":"unslothai/unsloth","slug":"training-job-was-superseded","errorCode":null,"errorMessage":"Training job was superseded","messagePattern":"Training job was superseded","errorType":"http","errorClass":"HTTPException","httpStatus":409,"severity":"info","filePath":"studio/backend/routes/training.py","lineNumber":2019,"sourceCode":"            event = \"training.status_failed\",\n            log = logger,\n        )\n\n\n@router.get(\"/metrics\", response_model = TrainingMetricsResponse)\nasync def get_training_metrics(\n    expected_job_id: Optional[str] = None, current_subject: str = Depends(get_current_subject)\n):\n    \"\"\"\n    Get training metrics (loss, learning rate, steps).\n    \"\"\"\n    try:\n        backend = get_training_backend()\n        job_id = getattr(backend, \"current_job_id\", \"\") or \"\"\n        if getattr(backend, \"_new_job_spawn_id\", None) is not None or (\n            expected_job_id is not None and expected_job_id != job_id\n        ):\n            raise HTTPException(status_code = 409, detail = \"Training job was superseded\")\n\n        loss_history = list(backend.loss_history)\n        lr_history = list(backend.lr_history)\n        step_history = list(backend.step_history)\n        grad_norm_history = list(getattr(backend, \"grad_norm_history\", []))\n        grad_norm_step_history = list(getattr(backend, \"grad_norm_step_history\", []))\n\n        if (\n            getattr(backend, \"_new_job_spawn_id\", None) is not None\n            or (getattr(backend, \"current_job_id\", \"\") or \"\") != job_id\n        ):\n            raise HTTPException(status_code = 409, detail = \"Training job was superseded\")\n\n        current_loss = loss_history[-1] if loss_history else None\n        current_lr = lr_history[-1] if lr_history else None\n        current_step = step_history[-1] if step_history else None\n\n        return TrainingMetricsResponse(","sourceCodeStart":2001,"sourceCodeEnd":2037,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/routes/training.py#L2001-L2037","documentation":"HTTP 409 from GET /training/metrics — first check: the metrics request is invalid because a new job spawn is in progress (backend._new_job_spawn_id set) or the optional expected_job_id does not match the backend's current_job_id. This guards clients against silently mixing metrics from two different runs.","triggerScenarios":"GET /training/metrics?expected_job_id=X where X differs from the running job's id, or any metrics fetch while a new training job is mid-spawn.","commonSituations":"A dashboard holding a job id from a previous run after a restart/resume spawned a new one; polling metrics immediately after issuing a start request.","solutions":["Refetch /training/status, take its job_id, and re-request metrics with that id.","If a spawn is in flight, wait for the start request to complete before polling metrics.","Treat this 409 as a signal to reset local metric history — the new run's series starts fresh."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"const status = await get('/training/status')\nif (!status.job_id || status.spawning) throw new Error('No stable job to read metrics for')\nconst metrics = await get(`/training/metrics?expected_job_id=${status.job_id}`)","typeGuard":null,"tryCatchPattern":"try { await get('/training/metrics', {params: {expected_job_id: jobId}}) } catch (e) { if (e.status === 409 && /superseded/.test(e.detail)) { const s = await get('/training/status'); resetLocalCharts(s.job_id) } else throw e }","preventionTips":["Always pass expected_job_id from your last successful status read.","On any 'superseded' 409, clear local metric history — mixing runs corrupts charts.","Do not poll metrics while a start request is still awaiting its response."],"tags":["metrics","superseded","job-id","http-409"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}