rqlite/rqlite · critical
list snapshots: %s
Error message
list snapshots: %s
What it means
During store.Open, rqlite lists pre-existing Raft snapshots via the snapshot store. If that List call fails, the open is aborted and the underlying error is wrapped as "list snapshots: %s". This means the node's snapshot directory could not be read, so previously persisted state cannot be validated before Raft starts.
Source
Thrown at store/store.go:609
}
if err := snapshot.Upgrade8To10(old8SnapshotDir, s.snapshotDir, s.logger); err != nil {
return fmt.Errorf("failed to upgrade v8 snapshots: %s", err)
}
// Create store for the Snapshots.
snapshotStore, err := snapshot.NewStore(s.snapshotDir)
if err != nil {
return fmt.Errorf("failed to create snapshot store: %s", err)
}
snapshotStore.SetNoVerifyDB(s.NoVerifyDB)
snapshotStore.LogReaping = s.hcLogLevel() < hclog.Warn
if s.SnapshotReapThreshold > 0 {
snapshotStore.SetReapThreshold(s.SnapshotReapThreshold)
}
s.snapshotStore = snapshotStore
snaps, err := s.snapshotStore.List()
if err != nil {
return fmt.Errorf("list snapshots: %s", err)
}
s.logger.Printf("%d preexisting snapshots present", len(snaps))
// Now, check if the most recent snapshot operation ran to completion
// without error and also check if the underlying DB file is unchanged since
// that snapshot. It shouldn't be changed -- that would require manual
// intervention which would potentially break rqlite -- but protect against
// it anyway. This could also happen in certain downgrade-then-upgrade-again
// scenarios. Anyway if it all looks good we can skip restoring the SQLite
// database from the Raft snapshot store because the contents are logically
// the same.
removeDBFiles := true
if err := func() error {
if snapshotStore.Len() == 0 {
return nil
}
defer func() {View on GitHub (pinned to 7586a4d1bd)
Solutions
- Check permissions on the node's data directory and its snapshots/ subdirectory (readable/writable by the rqlited user).
- Inspect the wrapped error in the log; if a specific snapshot file is corrupt, remove that snapshot file (keep the most recent valid one and the Raft log) and restart.
- Verify disk health and free space with df / dmesg.
- As a last resort, restore the node from a fresh backup or rejoin it to the cluster (delete the data dir and start with -join).
Example fix
// before $ ls -ld /data/node/snapshots drwx------ root root /data/node/snapshots // after $ chown -R rqlite:rqlite /data/node $ systemctl restart rqlited
Defensive patterns
Strategy: validation
Validate before calling
// before starting rqlited
const dataDir = "/data/node"
if err := os.MkdirAll(filepath.Join(dataDir, "snapshots"), 0o755); err != nil {
return err
}
if entries, err := os.ReadDir(filepath.Join(dataDir, "snapshots")); err != nil {
return fmt.Errorf("snapshot dir unreadable: %w", err)
} else {
_ = entries
} Try / catch
// supervisor: alert and restart only after fixing fs access
if strings.HasPrefix(err.Error(), "list snapshots:") {
log.Fatalf("snapshot dir unusable, halting restarts: %v", err)
} Prevention
- Run rqlited under a dedicated user with consistent ownership of the data dir
- Never chmod/chown the node directory while rqlited is running
- Monitor disk health and free space
- Copy backups, never the live data directory
When it happens
Trigger: s.snapshotStore.List() returns an error during Open — typically because the snapshots directory is unreadable, a snapshot file is corrupt or has bad permissions, or disk I/O fails.
Common situations: Restoring a data directory from a backup with wrong ownership/permissions; truncated snapshot files after a crash or full disk; running rqlited as a different user than the one that created the node directory.
Related errors
- failed to check for clean snapshot file: %s
- failed to determine size of Raft log: %s
- failed to recover node: %s
- snapshot integrity check failed: %s
- ErrEmptyFile
AI-assisted analysis of rqlite/rqlite@7586a4d1bd (2026-09-03).
Data as JSON: /api/errors/4b4974fe04aee6bb.
Report an issue: GitHub.