coollabsio/coolify · error · RuntimeException

The server is unavailable, so local backup archives cannot b

Error message

The server is unavailable, so local backup archives cannot be deleted.

What it means

Thrown when the schedule has execution rows with local_storage_deleted=false (backup archives still on the destination server) but $backup->server() resolves to null. The Server record behind the schedule is gone, so deleteBackupsLocally() has no host to clean up on. This is a data-integrity guard so local archives are not orphaned silently.

Source

Thrown at app/Actions/Shared/DeleteScheduledVolumeBackup.php:42

            if ($backup->executions()
                ->where(fn ($query) => $query
                    ->where('status', 'running')
                    ->orWhere('stop_recovery_pending', true)
                    ->orWhere('s3_cleanup_pending', true))
                ->exists()) {
                throw new \RuntimeException('Wait for the running storage backup and recovery operations to finish before deleting this schedule.');
            }

            $localFilenames = $backup->executions()
                ->where('local_storage_deleted', false)
                ->pluck('filename')
                ->filter()
                ->all();

            if ($localFilenames !== []) {
                $server ??= $backup->server();
                if (! $server) {
                    throw new \RuntimeException('The server is unavailable, so local backup archives cannot be deleted.');
                }

                deleteBackupsLocally($localFilenames, $server, throwError: true);
            }

            $s3Executions = $backup->executions()
                ->with('s3')
                ->where('s3_uploaded', true)
                ->where('s3_storage_deleted', false)
                ->get();

            foreach ($s3Executions->groupBy('s3_storage_id') as $executions) {
                $s3 = $executions->first()->s3;
                if (! $s3) {
                    throw new \RuntimeException('The S3 storage used by an existing backup is unavailable.');
                }

                $filenames = $executions->pluck('filename')->filter()->all();

View on GitHub (pinned to 70b9acc424)

Solutions

Pass an explicit $server as the second argument to the action if the archives live on a server you can still address.
If the machine is truly gone, manually remove the orphaned archive files (or accept they stay), then mark those executions local_storage_deleted=true or delete the execution rows, and retry.
Restore/recreate the Server record with the same id so the relation resolves, then let the action clean up normally.

Example fix

// before: throws 'The server is unavailable...'
DeleteScheduledVolumeBackup::run($backup);

// after: guard for an orphaned schedule before calling
if ($backup->executions()->where('local_storage_deleted', false)->exists() && is_null($backup->server())) {
    // archives are unrecoverable-by-Coolify; decide explicitly
    $backup->executions()->update(['local_storage_deleted' => true]);
}
DeleteScheduledVolumeBackup::run($backup);

Defensive patterns

Strategy: validation

Validate before calling

// Verify the server relation exists whenever local archives are undeleted
$hasLocalArchives = $backup->executions()->where('local_storage_deleted', false)->exists();

if ($hasLocalArchives && is_null($backup->server())) {
    // decide explicitly: pass a known $server, or acknowledge losing the files
    // DeleteScheduledVolumeBackup::run($backup, $server);
}

Type guard

function scheduleIsDeletable(\App\Models\ScheduledVolumeBackup $backup): bool
{
    $needsServer = $backup->executions()->where('local_storage_deleted', false)->exists();

    return ! $needsServer || ! is_null($backup->server());
}

Try / catch

try {
    DeleteScheduledVolumeBackup::run($backup);
} catch (\RuntimeException $e) {
    if (str_contains($e->getMessage(), 'server is unavailable')) {
        // surface a choice: point at another server, or mark archives abandoned
        report('Orphaned volume backup schedule '.$backup->id.' — server record missing');
        return;
    }
    throw $e;
}

Prevention

Delete a server's backup schedules before deleting the server in Coolify.
Include schedules in any server-deletion cleanup script (scheduled_volume_backups by server/database).
When passing $server explicitly, confirm SSH reachability first so local cleanup can actually run.

When it happens

Trigger: The Coolify Server record was deleted while a ScheduledVolumeBackup (or its executions with undeleted local files) still references it; the schedule outlived its server, so the relation returns null and undeleted local filenames exist.

Common situations: Team removes a decommissioned server from Coolify but forgets the volume backup schedules on it; database/server cleanup scripts delete servers without first deleting their backup schedules; restoring from a partial backup left orphaned schedules.

Related errors

AI-assisted analysis of coollabsio/coolify@70b9acc424 (2026-08-17). Data as JSON: /api/errors/182f3e77954cc8cb. Report an issue: GitHub.