{"record":{"id":"7892770301eae8ae","repo":"benbjohnson/litestream","slug":"read-hydrated-file-w","errorCode":null,"errorMessage":"read hydrated file: %w","messagePattern":"read hydrated file: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"vfs.go","lineNumber":884,"sourceCode":"\t\t}\n\n\t\toff := int64(phdr.Pgno-1) * int64(h.pageSize)\n\t\tif _, err := h.file.WriteAt(data, off); err != nil {\n\t\t\treturn fmt.Errorf(\"write page %d: %w\", phdr.Pgno, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// ReadAt reads data from the hydrated local file.\nfunc (h *Hydrator) ReadAt(p []byte, off int64) (int, error) {\n\th.mu.Lock()\n\tn, err := h.file.ReadAt(p, off)\n\th.mu.Unlock()\n\n\tif err != nil && err != io.EOF {\n\t\treturn n, fmt.Errorf(\"read hydrated file: %w\", err)\n\t}\n\n\t// Update the first page to pretend like we are in journal mode\n\tif off == 0 && len(p) >= 28 {\n\t\tp[18], p[19] = 0x01, 0x01\n\t\t_, _ = rand.Read(p[24:28])\n\t}\n\n\treturn n, nil\n}\n\n// ApplyUpdates fetches updated pages and writes them to the hydration file.\nfunc (h *Hydrator) ApplyUpdates(ctx context.Context, updates map[uint32]ltx.PageIndexElem) error {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\n\tfor pgno, elem := range updates {\n\t\t_, data, err := FetchPage(ctx, h.client, elem.Level, elem.MinTXID, elem.MaxTXID, elem.Offset, elem.Size)","sourceCodeStart":866,"sourceCodeEnd":902,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/vfs.go#L866-L902","documentation":"Hydrator.ReadAt wraps any non-EOF error from reading the local hydrated database file as \"read hydrated file\". Litestream's VFS serves SQLite reads from a locally hydrated copy of the database; if the underlying file read fails (bad fd, I/O error, out-of-range read on a closed/damaged file), the failure is wrapped with this message and returned to the SQLite layer.","triggerScenarios":"A VFS read is issued against the hydrated file while it is closed, deleted, or on a failing disk; a ReadAt at an offset beyond a truncated file that surfaces a non-EOF OS error; file descriptor exhaustion causing the pread to fail.","commonSituations":"Disk full or hardware I/O errors on the node hosting the hydration file; the hydration temp file being removed by tmp cleaners while in use; running out of file descriptors under heavy SQLite load; permissions changed on the hydration directory mid-run.","solutions":["Check the wrapped cause (%w) for os.PathError details — verify the hydration file path exists and is readable by the litestream process","Verify disk health and free space on the volume holding the hydration file (dmesg / SMART, df)","Check ulimit -n / file descriptor limits if errors appear under load","Use litestream reset for the database to clear corrupted local hydration state and re-hydrate from the replica"],"exampleFix":"// before\nn, err := h.file.ReadAt(p, off)\n// after\nif n, err = h.file.ReadAt(p, off); err != nil && err != io.EOF {\n\treturn n, fmt.Errorf(\"read hydrated file (off=%d, len=%d): %w\", off, len(p), err)\n}","handlingStrategy":"try-catch","validationCode":"// Go: before relying on the VFS, check the hydration file is accessible\nif fi, err := os.Stat(hydrationPath); err != nil || fi.Size() == 0 {\n\t// force re-hydration / litestream reset\n}","typeGuard":null,"tryCatchPattern":"n, err := v.ReadAt(buf, off)\nif err != nil {\n\tvar pe *os.PathError\n\tif errors.As(err, &pe) && errors.Is(pe.Err, syscall.ENOSPC) {\n\t\t// free disk space, then litestream reset\n\t}\n}","preventionTips":["Monitor free disk space on the hydration volume with alerting well before 100%","Raise ulimit -n for litestream under high SQLite concurrency","Exclude hydration temp paths from external cleanup jobs (tmpwatch, tmpreaper)","Use litestream reset proactively after any suspected disk I/O incident"],"tags":["io","filesystem","vfs","sqlite"],"backgroundTag":"file-read-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}