rqlite/rqlite · error

failed to persist snapshot: %v

Error message

failed to persist snapshot: %v

What it means

The recovery flow persists the full database state into the newly created Raft snapshot sink via fsmSnapshot.Persist(sink), which streams every row through the sink writer. This error means the write failed mid-stream — typically an I/O error writing the snapshot file or a failure reading the source database during streaming.

Source

Thrown at store/state.go:270

	if err != nil {
		return fmt.Errorf("failed to checkpoint database: %s", err)
	}
	streamer, err := snapshot.NewSnapshotStreamer(tmpDBPath)
	if err != nil {
		return fmt.Errorf("failed to create snapshot streamer: %s", err)
	}
	defer streamer.Close()
	if err := streamer.Open(); err != nil {
		return fmt.Errorf("failed to open snapshot streamer: %s", err)
	}
	fsmSnapshot := snapshot.NewStateReader(streamer) // tmpDBPath contains full state now.
	sink, err := snaps.Create(1, lastIndex, lastTerm, conf, 1, tn)
	if err != nil {
		return fmt.Errorf("failed to create snapshot: %v", err)
	}
	defer sink.Cancel() // If we fail, make sure to cancel the snapshot.
	if err = fsmSnapshot.Persist(sink); err != nil {
		return fmt.Errorf("failed to persist snapshot: %v", err)
	}
	if err = sink.Close(); err != nil {
		return fmt.Errorf("failed to finalize snapshot: %v", err)
	}
	logger.Printf("recovery snapshot %s created successfully using %s", sink.ID(), tmpDBPath)

	// Compact the log so that we don't get bad interference from any
	// configuration change log entries that might be there.
	firstLogIndex, err := logs.FirstIndex()
	if err != nil {
		return fmt.Errorf("failed to get first log index: %v", err)
	}
	if err := logs.DeleteRange(firstLogIndex, lastLogIndex); err != nil {
		return fmt.Errorf("log compaction failed: %v", err)
	}
	return nil
}

View on GitHub (pinned to 7586a4d1bd)

Solutions

1. Free disk space (or point the data dir at a larger volume) and re-run recovery
2. Check disk health (dmesg / smartctl) if I/O errors accompany the message
3. If a specific page read fails, the temp DB is corrupt — restore the node from a backup via /boot
4. Retry the recovery flow; the deferred sink.Cancel() removes partial snapshots so a clean retry is safe

// Pre-flight: enough free space for the snapshot (approx. DB size)
fi, err := os.Stat(tmpDBPath); if err != nil { return err }
var st syscall.Statfs_t
syscall.Statfs(dataDir, &st)
if st.Bavail*uint64(st.Bsize) < uint64(fi.Size())*2 {
    return errors.New("insufficient free space to persist snapshot")
}

err := store.RecoverNode(...)
if err != nil && strings.Contains(err.Error(), "failed to persist snapshot") {
    time.Sleep(retryDelay) // sink.Cancel() already cleaned the partial snapshot
    err = store.RecoverNode(...) // safe to retry after freeing space / transient I/O
}

When it happens

Trigger: fsmSnapshot.Persist(sink) returns an error during recovery: disk full while writing the snapshot file, the sink was cancelled/closed concurrently, or the SQLite streamer hit a read error partway through the dump.

Common situations: ENOSPC while snapshotting a large database; failing disk; DB corruption surfaced only when reading a particular page mid-dump; recovery interrupted by shutdown leaving a cancelled sink.

Related errors

• failed to open snapshot streamer: %s
• installed DB file is not a valid SQLite file
• installed WAL file is not a valid SQLite WAL file
• short write
• no WAL data available for snapshot

AI-assisted analysis of rqlite/rqlite@7586a4d1bd (2026-09-03). Data as JSON: /api/errors/13921d17dd60f7b7. Report an issue: GitHub.