What causes "Lease has been broken!" in phacility/phabricator?

Open the Drydock log for the broken lease (Drydock console > lease > logs) and read the underlying failure — the real cause is logged there, not in this exception.

phacility/phabricator · error · Exception

Lease has been broken!

Error message

Lease has been broken!

What it means

Wait-loop error in `bin/drydock lease` for status STATUS_BROKEN: the lease (or the resource backing it) entered a broken state before becoming active — typically because allocation or activation failed (blueprint threw, host provisioning failed, a command like git clone failed in activateResource). Broken is terminal for this lease attempt, so the CLI aborts instead of waiting forever.

Source

Thrown at src/applications/drydock/management/DrydockManagementLeaseWorkflow.php:252

        echo tsprintf(
          "(Lease #%d) <%s> %B\n",
          $lease->getID(),
          $type,
          $data);
      }

      $status = $lease->getStatus();

      switch ($status) {
        case DrydockLeaseStatus::STATUS_ACTIVE:
          $is_active = true;
          break;
        case DrydockLeaseStatus::STATUS_RELEASED:
          throw new Exception(pht('Lease has already been released!'));
        case DrydockLeaseStatus::STATUS_DESTROYED:
          throw new Exception(pht('Lease has already been destroyed!'));
        case DrydockLeaseStatus::STATUS_BROKEN:
          throw new Exception(pht('Lease has been broken!'));
        case DrydockLeaseStatus::STATUS_PENDING:
        case DrydockLeaseStatus::STATUS_ACQUIRED:
          break;
        default:
          throw new Exception(
            pht(
              'Lease has unknown status "%s".',
              $status));
      }

      if ($is_active) {
        break;
      } else {
        sleep(1);
      }
    }
  }

View on GitHub (pinned to 5720a38cfe)

Solutions

Open the Drydock log for the broken lease (Drydock console > lease > logs) and read the underlying failure — the real cause is logged there, not in this exception.
Fix the root cause (blueprint config, credentials, repository map, host capacity) and acquire a new lease.
If failures are transient (network blips during clone), simply retry the lease acquisition.

Defensive patterns

Strategy: try-catch

Validate before calling

if ($lease->getStatus() === DrydockLeaseStatus::STATUS_BROKEN) {
  // pull the real cause from the Drydock log before deciding
  $logs = id(new DrydockLogQuery())->setViewer($viewer)
    ->withLeasePHIDs(array($lease->getPHID()))->execute();
}

Try / catch

try {
  $lease->waitUntilActive();
} catch (Exception $ex) {
  if (preg_match('/has been broken/', $ex->getMessage())) {
    // inspect Drydock logs for the root cause; retry only if transient
    $cause = loadLatestDrydockLogForLease($lease);
    if (isTransientFailure($cause)) {
      $lease = acquireNewLease($attributes); // bounded single retry
    } else {
      throw $ex;
    }
  } else {
    throw $ex;
  }
}

Prevention

This exception is a symptom — always read the Drydock log entry for the broken lease before changing anything.
Validate blueprint inputs (repository PHIDs, VCS types, credentials) up front; most broken leases come from errors 540/541/544.
Bound retries: a systematically broken lease will break again until the config is fixed.

When it happens

Trigger: `bin/drydock lease --type working-copy` where blueprint activation fails (bad repository PHID, unsupported VCS, host lease failure, clone timeout) and the lease is marked broken; the polling loop then throws on the next status check.

Common situations: Blueprint misconfiguration (credentials, repositories.map entries per errors 540/541/544); host resources failing to provision; intermittent network failures during git clone/fetch breaking the lease.

Related errors

AI-assisted analysis of phacility/phabricator@5720a38cfe (2026-08-21). Data as JSON: /api/errors/328834f8bfd3e6b8. Report an issue: GitHub.