{"record":{"id":"dfd05e5f0d2b2052","repo":"phacility/phabricator","slug":"found-unexpected-task-status-s","errorCode":null,"errorMessage":"Found unexpected task status (\"%s\").","messagePattern":"Found unexpected task status \\(\"(.+?)\"\\)\\.","errorType":"exception","errorClass":"PhabricatorWorkerPermanentFailureException","httpStatus":null,"severity":"error","filePath":"src/infrastructure/daemon/workers/bulk/PhabricatorWorkerBulkJobTaskWorker.php","lineNumber":16,"sourceCode":"<?php\n\nfinal class PhabricatorWorkerBulkJobTaskWorker\n  extends PhabricatorWorkerBulkJobWorker {\n\n  protected function doWork() {\n    $lock = $this->acquireTaskLock();\n\n    $task = $this->loadTask();\n    $status = $task->getStatus();\n    switch ($task->getStatus()) {\n      case PhabricatorWorkerBulkTask::STATUS_WAITING:\n        // This is what we expect.\n        break;\n      default:\n        throw new PhabricatorWorkerPermanentFailureException(\n          pht(\n            'Found unexpected task status (\"%s\").',\n            $status));\n    }\n\n    $task\n      ->setStatus(PhabricatorWorkerBulkTask::STATUS_RUNNING)\n      ->save();\n\n    $lock->unlock();\n\n    $job = $this->loadJob();\n    $actor = $this->loadActor($job);\n\n    try {\n      $job->runTask($actor, $task);\n      $status = PhabricatorWorkerBulkTask::STATUS_DONE;\n    } catch (Exception $ex) {","sourceCodeStart":1,"sourceCodeEnd":34,"githubUrl":"https://github.com/phacility/phabricator/blob/5720a38cfe95b00ca4be5016dd0d2f3195f4fa04/src/infrastructure/daemon/workers/bulk/PhabricatorWorkerBulkJobTaskWorker.php#L1-L34","documentation":"PhabricatorWorkerBulkJobTaskWorker executes one unit of a bulk job and requires the bulk task row to still be in STATUS_WAITING when it starts (checked while holding a per-task global lock). A different status means the task already ran, is running elsewhere, or was completed — so the worker permanently fails instead of re-executing side effects.","triggerScenarios":"Duplicate task rows for one bulk task; a task re-queued after completion; a stale RUNNING row from a crashed daemon being executed again.","commonSituations":"Database restores that duplicate worker rows; aggressive manual use of bin/worker retry; crashed taskmasters leaving stale rows that later get re-executed after lease expiry.","solutions":["Check the bulk task's status; if it is RUNNING or DONE, the failure is the guard working — ignore the duplicate.","If a task is stuck in RUNNING after a crash, resolve the stale row via the worker management workflows before retrying.","Do not re-queue bulk task rows by hand."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  // execute the bulk task worker\n} catch (PhabricatorWorkerPermanentFailureException $ex) {\n  // terminal: check the bulk task status; ignore duplicates of DONE/RUNNING tasks\n}","preventionTips":["Never clone bulk task rows or use bin/worker retry on bulk tasks casually.","Resolve stale RUNNING rows via the worker management workflows after daemon crashes.","Treat this failure on duplicate rows as the guard working as designed."],"tags":["phabricator","bulk-jobs","worker-queue","race-condition"],"backgroundTag":"invalid-state-transition","analyzedSha":"5720a38cfe95b00ca4be5016dd0d2f3195f4fa04","analyzedAt":"2026-08-21T05:07:25.672Z","schemaVersion":2},"datasetVersion":"2026-08-21T11:28:35.574Z"}