can1357/oh-my-pi · critical · Error

All backups failed integrity check

Error message

All backups failed integrity check

What it means

emergencyRestore() iterates backups newest-first, attempting restoreBackup() on each. Failures are swallowed (matching Python recovery behavior) and the next backup is tried. If every candidate fails — corrupt archives or databases failing integrity checks — it throws this generic error after exhausting the list. The individual per-backup reasons are lost because the catch block discards errors.

Source

Thrown at packages/mnemopi/src/dr/recovery.ts:312

				.filter(name => /^mnemopi_backup_.*\.db\.gz$/.test(name))
				.sort()
				.reverse()
				.map(name => join(dir, name))
		: [];

	if (backups.length === 0) throw new FileNotFoundError(`No backups found in ${dir}`);

	let attempts = 0;
	for (const backup of backups) {
		attempts += 1;
		try {
			const result = restoreBackup(backup, targetPath);
			if (result.integrity_check) return { restored: true, backup_used: backup, attempts };
		} catch {
			// Try the next backup, matching the Python recovery behavior.
		}
	}
	throw new Error("All backups failed integrity check");
}
export function verifyIntegrity(dbPath?: string | null): boolean {
	const targetPath = dbPath ?? getDefaultPaths().dbPath;
	if (!existsSync(targetPath)) return false;

	let db: Database | null = null;
	try {
		db = openDatabase(targetPath, { create: false, readwrite: false, pragmas: false });
		const row = db.query("PRAGMA integrity_check").get() as { integrity_check: string } | null;
		return row?.integrity_check === "ok";
	} catch {
		return false;
	} finally {
		closeQuietly(db);
	}
}
export function listBackups(backupDir?: string | null): BackupInfo[] {
	const dir = backupDir ?? getDefaultPaths().backupDir;

View on GitHub (pinned to 9690622007)

Solutions

  1. Check disk space and filesystem health; free space or repair the volume, then retry.
  2. Manually test each backup (`gzip -t`, sqlite3 integrity_check) to find which fail and why.
  3. Produce a fresh backup from any surviving replica and place it in the backup dir.
  4. Restore from offsite/external backups, since all local candidates are unusable.

Example fix

// before
await emergencyRestore('/var/backups/mnemopi', dbPath);
// after
df -h /var/backups && gzip -t /var/backups/mnemopi_backup_*.db.gz  # diagnose first
try {
  await emergencyRestore('/var/backups/mnemopi', dbPath);
} catch (err) {
  console.error('All local backups failed; fetch offsite backup before retrying');
  throw err;
}
Defensive patterns

Strategy: try-catch

Validate before calling

import { readdirSync } from 'node:fs';
import { gunzipSync } from 'node:zlib';
import { readFileSync } from 'node:fs';
const candidates = readdirSync(dir).filter(n => /^mnemopi_backup_.*\.db\.gz$/.test(n));
const anyValid = candidates.some(n => {
  try {
    return gunzipSync(readFileSync(join(dir, n))).subarray(0, 16).toString('utf8').startsWith('SQLite format 3');
  } catch { return false; }
});
if (!anyValid) console.error('All local backups unreadable — fetch offsite copies first');

Try / catch

try {
  await emergencyRestore(dir, targetPath);
} catch (err) {
  if (err instanceof Error && err.message === 'All backups failed integrity check') {
    console.error('Every local backup is unusable; restore from offsite backup');
    // page on-call / start incident response — data recovery now depends on external copies
  } else throw err;
}

Prevention

When it happens

Trigger: Calling emergencyRestore(dir, targetPath) where every mnemopi_backup_*.db.gz in dir either fails to decompress/parse or fails verifyIntegrity during restoreBackup().

Common situations: A storage failure that corrupted the whole backup directory, all backups pruned to truncated files, restoring onto failing hardware, or gzip/SQLite version incompatibilities affecting every file.

Related errors

AI-assisted analysis of can1357/oh-my-pi@9690622007 (2026-08-31). Data as JSON: /api/errors/e50d2ce21b7187bc. Report an issue: GitHub.