{"record":{"id":"b29ab4752a6bf490","repo":"hashicorp/nomad","slug":"failed-to-read-after-v-v","errorCode":null,"errorMessage":"failed to read after %v: %v","messagePattern":"failed to read after (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"helper/snapshot/snapshot.go","lineNumber":246,"sourceCode":"\textra, err := io.ReadAll(decomp) // ReadAll consumes the EOF\n\tif err != nil {\n\t\treturn err\n\t} else if len(extra) != 0 {\n\t\treturn fmt.Errorf(\"%d unread uncompressed bytes remain\", len(extra))\n\t}\n\treturn nil\n}\n\ntype readWrapper struct {\n\tin io.Reader\n\tc  int\n}\n\nfunc (r *readWrapper) Read(b []byte) (int, error) {\n\tn, err := r.in.Read(b)\n\tr.c += n\n\tif err != nil && err != io.EOF {\n\t\treturn n, fmt.Errorf(\"failed to read after %v: %v\", r.c, err)\n\t}\n\treturn n, err\n}\n\n// Restore takes the snapshot from the reader and attempts to apply it to the\n// given Raft instance.\nfunc Restore(logger hclog.Logger, in io.Reader, r *raft.Raft) error {\n\t// Wrap the reader in a gzip decompressor.\n\tdecomp, err := gzip.NewReader(&readWrapper{in, 0})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to decompress snapshot: %v\", err)\n\t}\n\tdefer func() {\n\t\tif err := decomp.Close(); err != nil {\n\t\t\tlogger.Error(\"Failed to close snapshot decompressor\", \"error\", err)\n\t\t}\n\t}()\n","sourceCodeStart":228,"sourceCodeEnd":264,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/snapshot/snapshot.go#L228-L264","documentation":"Restore wraps the incoming reader so every read error after io.EOF's byte count is annotated with the total bytes read so far. Any non-EOF read error while streaming the snapshot into the restorer is reported as 'failed to read after N bytes: <err>', which pinpoints how far into the snapshot the failure occurred.","triggerScenarios":"Restore (via snapshotRestore) where r.in.Read returns a non-EOF error mid-stream: network connection reset while downloading the snapshot, TLS errors, storage read failures on a local snapshot file, or timeouts closing the body.","commonSituations":"Long snapshots over flaky connections hit read timeouts; proxy closes idle connections mid-transfer; snapshot file on degraded storage returning EIO partway through.","solutions":["Use the N-bytes position to compare against the expected snapshot size and confirm where truncation happened.","Retry the restore with a fresh download; transient network errors usually clear.","Ensure the source (HTTP client/CLI) uses streaming with adequate timeouts and connection keep-alive for large snapshots.","Verify the local snapshot file's integrity (checksum) if reading from disk — failing storage is the likely cause otherwise."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// check expected size is available before restore\nfunc expectSize(rc io.ReadCloser, want int64) io.ReadCloser { return sizeCheck{rc, want, 0} }\n// abort early if the stream ends well before the expected snapshot size","typeGuard":null,"tryCatchPattern":"err := snapshot.Restore(logger, in, r)\nif err != nil && strings.Contains(err.Error(), \"failed to read after\") {\n\t// transient network errors: safe to retry — state untouched if read failed\n\treturn retryWithBackoff(func() error { return redownloadAndRestore() })\n}","preventionTips":["Use generous client timeouts and keep-alive for large snapshot transfers.","Retry restores on transient read errors; Raft state is not applied until the stream completes.","Verify checksums after download and before restore to fail fast.","Avoid reading snapshot files from degraded or network storage."],"tags":["network","io","snapshot","stream"],"backgroundTag":"read-failure-mid-stream","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}