{"record":{"id":"f22067b07eea6c77","repo":"unslothai/unsloth","slug":"job-already-running","errorCode":null,"errorMessage":"job already running","messagePattern":"job already running","errorType":"exception","errorClass":"RuntimeError","httpStatus":409,"severity":"error","filePath":"studio/backend/core/data_recipe/jobs/manager.py","lineNumber":155,"sourceCode":"        ``internal_api_key_id`` is a workflow-scoped sk-unsloth-* key row id\n        minted by the route layer; revoked on terminal state so the key's\n        live window is no longer than the run.\n        \"\"\"\n        llm_columns = recipe.get(\"columns\") or []\n        llm_column_count = 0\n        if isinstance(llm_columns, list):\n            for column in llm_columns:\n                if not isinstance(column, dict):\n                    continue\n                column_type = str(column.get(\"column_type\") or \"\").strip().lower()\n                if column_type.startswith(\"llm\"):\n                    llm_column_count += 1\n        if llm_column_count <= 0:\n            llm_column_count = 1\n\n        with self._lock:\n            if self._proc is not None and self._proc.is_alive():\n                raise RuntimeError(\"job already running\")\n\n            job_id = uuid.uuid4().hex\n            self._job = Job(job_id = job_id, status = \"pending\", started_at = time.time())\n            self._job.progress_columns_total = llm_column_count\n            self._job.source_progress_estimated_total = _github_source_estimated_total(recipe)\n            self._job.internal_api_key_id = internal_api_key_id\n            self._events.clear()\n            self._seq = 0\n\n            run_payload = dict(run)\n            run_payload[\"_job_id\"] = job_id\n            from utils.native_path_leases import (\n                native_path_secret_removed_for_child_start,\n                run_without_native_path_secret,\n            )\n            from utils.hf_cache_settings import child_environment_for_spawn, get_hf_cache_paths\n\n            cache_env = get_hf_cache_paths().child_env({})","sourceCodeStart":137,"sourceCodeEnd":173,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/data_recipe/jobs/manager.py#L137-L173","documentation":"RuntimeError raised under the job manager's lock when starting a new recipe job while self._proc is still alive: the manager is a single-process job runner (one Job, one _proc, one event buffer), so a second concurrent start is refused rather than queued. The check happens after event/job state would otherwise be reset, protecting the running job's state from being clobbered.","triggerScenarios":"Calling the job start/run endpoint twice in quick succession (double-click, client retry without idempotency); starting a new recipe while a previous long-running recipe job (with LLM columns) is still executing; a zombie child process that is_alive() but stuck.","commonSituations":"Frontend double-submits; background tab or script retrying on timeout while the first request actually started the job; long LLM-generation jobs where users assume the first attempt failed.","solutions":["Query the current job status first and only start when no job is running (or surface 'already running' to the user as a stop-and-restart prompt).","Debounce/disable the Run button in the UI while a job is active.","Make client retries idempotent: do not blindly re-POST the start endpoint on timeout.","If the process is truly stuck, stop/cancel the existing job (or restart the backend) before starting a new one."],"exampleFix":"# before\nmanager.start(...)  # may raise 'job already running'\n\n# after\nstatus = manager.status()\nif status and status.get('running'):\n    raise_or_prompt('A job is already running; stop it first')\nmanager.start(...)","handlingStrategy":"validation","validationCode":"job = manager.status()\nif job is not None and job.get('status') in ('pending', 'running'):\n    raise BusyError('stop the running job before starting another')","typeGuard":null,"tryCatchPattern":"try:\n    manager.start(...)\nexcept RuntimeError as e:\n    if 'job already running' in str(e):\n        offer_stop_and_restart(); return","preventionTips":["Disable/debounce the Run control while a job is active.","Make start requests idempotent in automation; never blind-retry them.","Poll job status before attempting a new start."],"tags":["concurrency","job-management","data-recipe","studio"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}