{"record":{"id":"13921d17dd60f7b7","repo":"rqlite/rqlite","slug":"failed-to-persist-snapshot-v","errorCode":null,"errorMessage":"failed to persist snapshot: %v","messagePattern":"failed to persist snapshot: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"store/state.go","lineNumber":270,"sourceCode":"\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to checkpoint database: %s\", err)\n\t}\n\tstreamer, err := snapshot.NewSnapshotStreamer(tmpDBPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to create snapshot streamer: %s\", err)\n\t}\n\tdefer streamer.Close()\n\tif err := streamer.Open(); err != nil {\n\t\treturn fmt.Errorf(\"failed to open snapshot streamer: %s\", err)\n\t}\n\tfsmSnapshot := snapshot.NewStateReader(streamer) // tmpDBPath contains full state now.\n\tsink, err := snaps.Create(1, lastIndex, lastTerm, conf, 1, tn)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to create snapshot: %v\", err)\n\t}\n\tdefer sink.Cancel() // If we fail, make sure to cancel the snapshot.\n\tif err = fsmSnapshot.Persist(sink); err != nil {\n\t\treturn fmt.Errorf(\"failed to persist snapshot: %v\", err)\n\t}\n\tif err = sink.Close(); err != nil {\n\t\treturn fmt.Errorf(\"failed to finalize snapshot: %v\", err)\n\t}\n\tlogger.Printf(\"recovery snapshot %s created successfully using %s\", sink.ID(), tmpDBPath)\n\n\t// Compact the log so that we don't get bad interference from any\n\t// configuration change log entries that might be there.\n\tfirstLogIndex, err := logs.FirstIndex()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to get first log index: %v\", err)\n\t}\n\tif err := logs.DeleteRange(firstLogIndex, lastLogIndex); err != nil {\n\t\treturn fmt.Errorf(\"log compaction failed: %v\", err)\n\t}\n\treturn nil\n}\n","sourceCodeStart":252,"sourceCodeEnd":288,"githubUrl":"https://github.com/rqlite/rqlite/blob/7586a4d1bdbd9a5a80021664c5a863cd850adb60/store/state.go#L252-L288","documentation":"The recovery flow persists the full database state into the newly created Raft snapshot sink via fsmSnapshot.Persist(sink), which streams every row through the sink writer. This error means the write failed mid-stream — typically an I/O error writing the snapshot file or a failure reading the source database during streaming.","triggerScenarios":"fsmSnapshot.Persist(sink) returns an error during recovery: disk full while writing the snapshot file, the sink was cancelled/closed concurrently, or the SQLite streamer hit a read error partway through the dump.","commonSituations":"ENOSPC while snapshotting a large database; failing disk; DB corruption surfaced only when reading a particular page mid-dump; recovery interrupted by shutdown leaving a cancelled sink.","solutions":["Free disk space (or point the data dir at a larger volume) and re-run recovery","Check disk health (dmesg / smartctl) if I/O errors accompany the message","If a specific page read fails, the temp DB is corrupt — restore the node from a backup via /boot","Retry the recovery flow; the deferred sink.Cancel() removes partial snapshots so a clean retry is safe"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Pre-flight: enough free space for the snapshot (approx. DB size)\nfi, err := os.Stat(tmpDBPath); if err != nil { return err }\nvar st syscall.Statfs_t\nsyscall.Statfs(dataDir, &st)\nif st.Bavail*uint64(st.Bsize) < uint64(fi.Size())*2 {\n    return errors.New(\"insufficient free space to persist snapshot\")\n}","typeGuard":null,"tryCatchPattern":"err := store.RecoverNode(...)\nif err != nil && strings.Contains(err.Error(), \"failed to persist snapshot\") {\n    time.Sleep(retryDelay) // sink.Cancel() already cleaned the partial snapshot\n    err = store.RecoverNode(...) // safe to retry after freeing space / transient I/O\n}","preventionTips":["Keep at least 2x DB size free on the data volume during recovery","Monitor disk health (SMART) on nodes holding rqlite data","Don't shut down rqlited mid-recovery; let the snapshot finish","If persistence fails repeatedly, switch to /boot from a clean backup"],"tags":["snapshot","io","sqlite","recovery"],"backgroundTag":"snapshot-persist-failed","analyzedSha":"7586a4d1bdbd9a5a80021664c5a863cd850adb60","analyzedAt":"2026-09-03T07:03:02.260Z","contentChangedAt":"2026-09-03T07:03:02.260Z","schemaVersion":2},"datasetVersion":"2026-09-10T12:17:11.382Z"}