phacility/phabricator · error · PhabricatorWorkerPermanentFailureException

Lease "%s" could not be loaded.

Error message

Lease "%s" could not be loaded.

What it means

The 'Lease Working Copy' step stores the acquired Drydock lease PHID in the target detail 'exec.leasePHID'. When the target runs again, it re-loads the lease by PHID; if the lease no longer exists (destroyed, released, or garbage-collected) this PhabricatorWorkerPermanentFailureException is thrown, permanently failing the worker task and the build target.

Source

Thrown at src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php:35

  public function execute(
    HarbormasterBuild $build,
    HarbormasterBuildTarget $build_target) {
    $viewer = PhabricatorUser::getOmnipotentUser();

    $settings = $this->getSettings();

    // TODO: We should probably have a separate temporary storage area for
    // execution stuff that doesn't step on configuration state?
    $lease_phid = $build_target->getDetail('exec.leasePHID');

    if ($lease_phid) {
      $lease = id(new DrydockLeaseQuery())
        ->setViewer($viewer)
        ->withPHIDs(array($lease_phid))
        ->executeOne();
      if (!$lease) {
        throw new PhabricatorWorkerPermanentFailureException(
          pht(
            'Lease "%s" could not be loaded.',
            $lease_phid));
      }
    } else {
      $working_copy_type = id(new DrydockWorkingCopyBlueprintImplementation())
        ->getType();

      $allowed_phids = $build_target->getFieldValue('blueprintPHIDs');
      if (!is_array($allowed_phids)) {
        $allowed_phids = array();
      }
      $authorizing_phid = $build_target->getBuildStep()->getPHID();

      $lease = DrydockLease::initializeNewLease()
        ->setResourceType($working_copy_type)
        ->setOwnerPHID($build_target->getPHID())
        ->setAuthorizingPHID($authorizing_phid)

View on GitHub (pinned to 5720a38cfe)

Solutions

1. Restart the build (or re-run the target) so the step acquires a fresh working-copy lease instead of reusing the dead PHID.
2. Check the Drydock lease and resource logs for why the lease was destroyed (expired, reclaimed, blueprint failure).
3. Increase lease/resource lifetime or reduce reclaim aggressiveness so leases survive worker retry delays.
4. Avoid manually releasing Drydock leases while Harbormaster builds are in progress.

// Before re-running a target that already has a lease detail:
$lease_phid = $build_target->getDetail('exec.leasePHID');
if ($lease_phid) {
  $lease = id(new DrydockLeaseQuery())
    ->setViewer(PhabricatorUser::getOmnipotentUser())
    ->withPHIDs(array($lease_phid))
    ->executeOne();
  if (!$lease) {
    // lease is gone - restart the build to acquire a fresh one
  }
}

try {
  // execute lease step
} catch (PhabricatorWorkerPermanentFailureException $ex) {
  if (preg_match('/could not be loaded/', $ex->getMessage())) {
    // permanent: do not retry the same target; restart the build
  }
}

When it happens

Trigger: A build target yielded or retried after saving 'exec.leasePHID' (e.g. lease still activating), and in the meantime the lease was released or expired and was destroyed by Drydock. Also triggered by manually releasing Drydock leases while builds are pending.

Common situations: Long-running builds outliving lease lifetime; aggressive Drydock resource reclaim settings; an administrator releasing leases from the Drydock console during a build; daemons backed up so retries happen much later.

Related errors

• Lease "%s" never activated.
• Working copy lease is missing required attribute "%s". Attr
• Use "--lease" to specify a lease.
• Unable to load lease with ID "%s"!
• Lease has already been released!

AI-assisted analysis of phacility/phabricator@5720a38cfe (2026-08-21). Data as JSON: /api/errors/88c384e5c7e86c57. Report an issue: GitHub.