phacility/phabricator · error · PhutilProxyException

Unexpected failure while destroying resource ("%s").

Error message

Unexpected failure while destroying resource ("%s").

What it means

breakResource() is the resource update worker's failure handler: when processing a resource throws, it marks the resource BROKEN. If the resource is already broken, released, or destroyed, there is nothing left to break, so the original exception is wrapped in a PhutilProxyException with this message. As the comment notes, this plain exception makes the task retry later; the nested exception carries the real cause.

Source

Thrown at src/applications/drydock/worker/DrydockResourceUpdateWorker.php:269

    $this->destroyResource($resource);
  }


/* -(  Breaking Resources  )------------------------------------------------- */


  /**
   * @task break
   */
  private function breakResource(DrydockResource $resource, Exception $ex) {
    switch ($resource->getStatus()) {
      case DrydockResourceStatus::STATUS_BROKEN:
      case DrydockResourceStatus::STATUS_RELEASED:
      case DrydockResourceStatus::STATUS_DESTROYED:
        // If the resource was already broken, just throw a normal exception.
        // This will retry the task eventually.
        throw new PhutilProxyException(
          pht(
            'Unexpected failure while destroying resource ("%s").',
            $resource->getPHID()),
          $ex);
    }

    $resource
      ->setStatus(DrydockResourceStatus::STATUS_BROKEN)
      ->save();

    $resource->scheduleUpdate();

    $resource->logEvent(
      DrydockResourceActivationFailureLogType::LOGCONST,
      array(
        'class' => get_class($ex),
        'message' => $ex->getMessage(),
      ));

View on GitHub (pinned to 5720a38cfe)

Solutions

1. Inspect the proxied original exception to find the actual cleanup failure.
2. Make the blueprint's destroy logic tolerant of partially-destroyed resources (idempotent cleanup).
3. If the resource is a zombie (broken but undeletable), finish the destroy manually via Drydock CLI after fixing the underlying error.

if (in_array($resource->getStatus(), array(
  DrydockResourceStatus::STATUS_BROKEN,
  DrydockResourceStatus::STATUS_RELEASED,
  DrydockResourceStatus::STATUS_DESTROYED,
))) {
  return; // already finalized; skip
}

try {
  $this->destroyResource($resource);
} catch (PhutilProxyException $ex) {
  $root = $ex->getPrevious_exception ? $ex->getPreviousException() : $ex;
  phlog($root->getMessage()); // the real cleanup failure
}

When it happens

Trigger: A destroy/update task fails (blueprint destroyResource throws, remote host unreachable) while the resource is already STATUS_BROKEN/RELEASED/DESTROYED; two workers failing the same resource concurrently, the second one hitting an already-broken row.

Common situations: Destruction of a broken resource failing because the host is gone; SSH or permission errors during cleanup; retried resource update tasks after the resource was already broken.

Related errors

• Unexpected failure while destroying lease ("%s").
• Trying to activate a resource which has not yet been persist
• Trying to activate a resource from the wrong status. Status
• Blueprint "%s" (of type "%s") is not properly implemented: %
• Trying to activate lease ("%s") on a resource ("%s") in the

AI-assisted analysis of phacility/phabricator@5720a38cfe (2026-08-21). Data as JSON: /api/errors/47b92da37b79fae2. Report an issue: GitHub.