phacility/phabricator · error · Exception

Lease has been broken!

Error message

Lease has been broken!

What it means

Wait-loop error in `bin/drydock lease` for status STATUS_BROKEN: the lease (or the resource backing it) entered a broken state before becoming active — typically because allocation or activation failed (blueprint threw, host provisioning failed, a command like git clone failed in activateResource). Broken is terminal for this lease attempt, so the CLI aborts instead of waiting forever.

Solutions

1. Open the Drydock log for the broken lease (Drydock console > lease > logs) and read the underlying failure — the real cause is logged there, not in this exception.
2. Fix the root cause (blueprint config, credentials, repository map, host capacity) and acquire a new lease.
3. If failures are transient (network blips during clone), simply retry the lease acquisition.

if ($lease->getStatus() === DrydockLeaseStatus::STATUS_BROKEN) {
  // pull the real cause from the Drydock log before deciding
  $logs = id(new DrydockLogQuery())->setViewer($viewer)
    ->withLeasePHIDs(array($lease->getPHID()))->execute();
}

try {
  $lease->waitUntilActive();
} catch (Exception $ex) {
  if (preg_match('/has been broken/', $ex->getMessage())) {
    // inspect Drydock logs for the root cause; retry only if transient
    $cause = loadLatestDrydockLogForLease($lease);
    if (isTransientFailure($cause)) {
      $lease = acquireNewLease($attributes); // bounded single retry
    } else {
      throw $ex;
    }
  } else {
    throw $ex;
  }
}

When it happens

Trigger: `bin/drydock lease --type working-copy` where blueprint activation fails (bad repository PHID, unsupported VCS, host lease failure, clone timeout) and the lease is marked broken; the polling loop then throws on the next status check.

Common situations: Blueprint misconfiguration (credentials, repositories.map entries per errors 540/541/544); host resources failing to provision; intermittent network failures during git clone/fetch breaking the lease.

Related errors

• Lease has already been destroyed!
• Lease has already been released!
• Failed to acquire lock for resource
• Lease has unknown status
• Lease " " could not be loaded.

AI-assisted analysis of phacility/phabricator@5720a38cfe (2026-08-21). Data as JSON: /api/errors/328834f8bfd3e6b8. Report an issue: GitHub.

Appendix: source

Thrown at src/applications/drydock/management/DrydockManagementLeaseWorkflow.php:252

        echo tsprintf(
          "(Lease #%d) <%s> %B\n",
          $lease->getID(),
          $type,
          $data);
      }

      $status = $lease->getStatus();

      switch ($status) {
        case DrydockLeaseStatus::STATUS_ACTIVE:
          $is_active = true;
          break;
        case DrydockLeaseStatus::STATUS_RELEASED:
          throw new Exception(pht('Lease has already been released!'));
        case DrydockLeaseStatus::STATUS_DESTROYED:
          throw new Exception(pht('Lease has already been destroyed!'));
        case DrydockLeaseStatus::STATUS_BROKEN:
          throw new Exception(pht('Lease has been broken!'));
        case DrydockLeaseStatus::STATUS_PENDING:
        case DrydockLeaseStatus::STATUS_ACQUIRED:
          break;
        default:
          throw new Exception(
            pht(
              'Lease has unknown status "%s".',
              $status));
      }

      if ($is_active) {
        break;
      } else {
        sleep(1);
      }
    }
  }

View on GitHub (pinned to 5720a38cfe)