{"record":{"id":"73c356e6111ccff2","repo":"benbjohnson/litestream","slug":"write-wal-segment-d-d-w","errorCode":null,"errorMessage":"write WAL segment %d/%d: %w","messagePattern":"write WAL segment (.+?)/(.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"replica.go","lineNumber":1297,"sourceCode":"\t\tif seg.Offset == 0 {\n\t\t\t// Apply the last WAL file, if any\n\t\t\tif err = applyLastWalFile(); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif seg.Index != expectedIndex {\n\t\t\t\treturn fmt.Errorf(\"missing WAL index: expected %d/0, got %d/%d\", expectedIndex, seg.Index, seg.Offset)\n\t\t\t}\n\t\t\toffset = 0\n\t\t\t// Open a new WAL file\n\t\t\tif f, err = os.OpenFile(walPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0644); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\texpectedIndex++\n\t\t} else if seg.Offset != offset {\n\t\t\treturn fmt.Errorf(\"missing WAL segment: expected %d/%d, got %d/%d\", seg.Index, offset, seg.Index, seg.Offset)\n\t\t}\n\t\tif n, err := r.appendWALSegmentV3(ctx, client, generation, seg, f); err != nil {\n\t\t\treturn fmt.Errorf(\"write WAL segment %d/%d: %w\", seg.Index, seg.Offset, err)\n\t\t} else {\n\t\t\toffset += n\n\t\t\tr.Logger().Debug(\"wrote WAL segment\", \"generation\", generation, \"index\", seg.Index, \"offset\", seg.Offset, \"bytes\", n)\n\t\t}\n\t}\n\n\treturn applyLastWalFile()\n}\n\n// appendWALSegmentV3 appends the specified segment to an open WAL file f.\nfunc (r *Replica) appendWALSegmentV3(ctx context.Context, client ReplicaClientV3, generation string, seg WALSegmentInfoV3, f *os.File) (int64, error) {\n\t// Download WAL segment.\n\trc, err := client.OpenWALSegmentV3(ctx, generation, seg.Index, seg.Offset)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tdefer func() { _ = rc.Close() }()\n","sourceCodeStart":1279,"sourceCodeEnd":1315,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/replica.go#L1279-L1315","documentation":"Wraps any error from appendWALSegmentV3, which downloads one v0.3.x WAL segment from the replica client and appends it to the local WAL file being reconstructed during restore. The message identifies the failing segment as index/offset. The underlying cause is a download/IO failure, not a logical gap.","triggerScenarios":"RestoreV3's replay loop calls r.appendWALSegmentV3(ctx, client, generation, seg, f) and it fails: network error fetching the segment from S3/GS/Azure/local storage, permission error opening the object, or write failure to the local tmp WAL file (disk full).","commonSituations":"Transient cloud-storage outages or throttling during restore; expired/missing storage credentials; local disk full while reconstructing a large WAL; proxy/firewall interrupting long restores.","solutions":["Read the wrapped cause: network/403/404 from the backend vs local write error need different fixes.","Retry the restore — segment downloads are idempotent and transient network errors are common.","Verify storage credentials and object permissions for the bucket/prefix.","Free disk space where the output path lives; the reconstructed WAL can be large.","If a specific object is persistently missing/unreadable, restore from another generation."],"exampleFix":"// before\nos.Setenv(\"AWS_ACCESS_KEY_ID\", \"\")\n// after\nexport AWS_ACCESS_KEY_ID=... && litestream restore -output db.sqlite /path/db","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"var err error\nfor i := 0; i < 3; i++ {\n    if err = replica.Restore(ctx, opt); err == nil ||\n        !strings.Contains(err.Error(), \"write WAL segment\") {\n        break\n    }\n    time.Sleep(backoff(i))\n}","preventionTips":["Ensure storage credentials are valid and objects are readable before restoring","Reserve ample disk space for snapshot + reconstructed WAL","Use stable network paths; avoid restoring over flaky VPNs","Apply exponential backoff on transient backend errors"],"tags":["wal","download","network","storage"],"backgroundTag":"network-request-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}