coollabsio/coolify · error · Exception

SSH key file content verification failed: {$keyLocation}

Error message

SSH key file content verification failed: {$keyLocation}

What it means

Thrown by PrivateKey::storeInFileSystem() (app/Models/PrivateKey.php:240). After put() and exists() succeed, the routine reads the file back and compares it byte-for-byte with $this->private_key. A mismatch (or empty read) means the file on disk does not equal what was just written under an exclusive lock — indicating concurrent writers bypassing the lock (e.g. a process writing the same ssh_key@{uuid} path directly), disk corruption, or storage layer caching (network/FUSE). The bad file is deleted before throwing, so no corrupt key is left behind.

Source

Thrown at app/Models/PrivateKey.php:240

                throw new \Exception("Failed to acquire lock for SSH key: {$keyLocation}");
            }

            // Attempt to store the private key
            $success = $disk->put($filename, $this->private_key);

            if (! $success) {
                throw new \Exception("Failed to write SSH key to filesystem. Check disk space and permissions for: {$keyLocation}");
            }

            // Verify the file was actually created and has content
            if (! $disk->exists($filename)) {
                throw new \Exception("SSH key file was not created: {$keyLocation}");
            }

            $storedContent = $disk->get($filename);
            if (empty($storedContent) || $storedContent !== $this->private_key) {
                $disk->delete($filename); // Clean up the bad file
                throw new \Exception("SSH key file content verification failed: {$keyLocation}");
            }

            // Ensure correct permissions for SSH (0600 required)
            if (file_exists($keyLocation) && ! chmod($keyLocation, 0600)) {
                Log::warning('Failed to set SSH key file permissions to 0600', [
                    'key_uuid' => $this->uuid,
                    'path' => $keyLocation,
                ]);
            }

            return $keyLocation;
        } finally {
            flock($lockHandle, LOCK_UN);
            fclose($lockHandle);
        }
    }

    public static function deleteFromStorage(self $privateKey)

View on GitHub (pinned to 70b9acc424)

Solutions

  1. Verify only one Coolify instance/worker writes to this ssh-keys storage; split data dirs per instance.
  2. Move storage off network filesystems to local disk (the read-back compare assumes coherent local IO).
  3. Retry the save — the corrupt file was already deleted by the guard.
Defensive patterns

Strategy: retry

Try / catch

try {
    $privateKey->storeInFileSystem();
} catch (\Exception $e) {
    if (str_contains($e->getMessage(), 'content verification failed')) {
        // corrupt file was auto-deleted; rule out concurrent writers (second instance / host scripts), then retry
    }
    throw $e;
}

Prevention

When it happens

Trigger: Two workers storing different key material under the same uuid simultaneously; a host-side process modifying files in /data/coolify/ssh; silently corrupting network storage; a model whose private_key attribute changed between put and the read-back.

Common situations: Duplicate Coolify control planes sharing one data dir; scripts editing key files on the host; flaky NFS/FUSE caching returning stale content on immediate re-read.

Related errors


AI-assisted analysis of coollabsio/coolify@70b9acc424 (2026-08-17). Data as JSON: /api/errors/b80bea36376fcbf3. Report an issue: GitHub.