{"record":{"id":"3fb280f7419d43e3","repo":"benbjohnson/litestream","slug":"list-level-d-ltx-files-w","errorCode":null,"errorMessage":"list level %d ltx files: %w","messagePattern":"list level (.+?) ltx files: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"replica.go","lineNumber":1010,"sourceCode":"\t\t}\n\t}\n\n\tif err := dec.Close(); err != nil {\n\t\treturn fmt.Errorf(\"close decoder: %w\", err)\n\t}\n\n\treturn f.Sync()\n}\n\n// fillFollowGap attempts to bridge a gap in level 0 files by searching\n// higher compaction levels for a file that covers the missing TXID range.\nfunc (r *Replica) fillFollowGap(ctx context.Context, f *os.File, afterTXID ltx.TXID, gapMinTXID ltx.TXID, pageSize uint32) (ltx.TXID, error) {\n\tcurrentTXID := afterTXID\n\n\tfor level := 1; level < SnapshotLevel; level++ {\n\t\titr, err := r.Client.LTXFiles(ctx, level, 0, false)\n\t\tif err != nil {\n\t\t\treturn currentTXID, fmt.Errorf(\"list level %d ltx files: %w\", level, err)\n\t\t}\n\t\tcloseLevel := func(retErr error) (ltx.TXID, error) {\n\t\t\tif closeErr := itr.Close(); closeErr != nil {\n\t\t\t\tcloseErr = fmt.Errorf(\"close level %d ltx iterator: %w\", level, closeErr)\n\t\t\t\tif retErr != nil {\n\t\t\t\t\treturn currentTXID, errors.Join(retErr, closeErr)\n\t\t\t\t}\n\t\t\t\treturn currentTXID, closeErr\n\t\t\t}\n\t\t\treturn currentTXID, retErr\n\t\t}\n\n\t\tfor itr.Next() {\n\t\t\tinfo := itr.Item()\n\n\t\t\t// Skip if there's a gap at this level too.\n\t\t\tif info.MinTXID > currentTXID+1 {\n\t\t\t\tbreak","sourceCodeStart":992,"sourceCodeEnd":1028,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/replica.go#L992-L1028","documentation":"fillFollowGap, which tries to bridge a TXID gap in level-0 files by scanning compaction levels 1..SnapshotLevel-1, failed while listing the LTX files for a level via the storage client (r.Client.LTXFiles). The wrapped error comes from the storage backend, not from local files.","triggerScenarios":"r.Client.LTXFiles(ctx, level, 0, false) returns an error for levels 1-3: storage API failure listing objects, expired/invalid credentials, network failure, or a misconfigured bucket/container.","commonSituations":"S3/GCS/Azure credential expiry or permission loss, rate limiting on ListObjects, region or bucket name misconfiguration, network outage during replica follow mode.","solutions":["Verify storage credentials and permissions for listing objects in the bucket/container","Check network connectivity and proxy settings to the storage endpoint","Confirm bucket/region/container configuration in the litestream config file","Check for storage-side rate limits (429s) and add backoff/reduce polling frequency"],"exampleFix":"// before: missing s3:ListBucket permission\n// policy lacks s3:ListBucket on the bucket\n// after: grant it\n{ \"Effect\": \"Allow\", \"Action\": [\"s3:GetObject\", \"s3:PutObject\", \"s3:ListBucket\"], \"Resource\": [\"arn:aws:s3:::my-bucket\", \"arn:aws:s3:::my-bucket/*\"] }","handlingStrategy":"retry","validationCode":"// preflight: can we list the replica path?\nitr, err := client.LTXFiles(ctx, 1, 0, false)\nif err != nil {\n    return fmt.Errorf(\"storage preflight failed (check credentials/bucket): %w\", err)\n}\nitr.Close()","typeGuard":null,"tryCatchPattern":"if err := r.applyNewLTXFiles(ctx, f, pageSize); err != nil {\n    var storageErr *StorageError\n    if errors.As(err, &storageErr) || strings.Contains(err.Error(), \"list level\") {\n        // transient listing failure: retry with backoff\n        return retryWithBackoff(3, func() error { return r.applyNewLTXFiles(ctx, f, pageSize) })\n    }\n    return err\n}","preventionTips":["Grant s3:ListBucket (or equivalent) on the replica bucket/prefix","Rotate credentials before expiry; use instance roles where possible","Set explicit region/endpoint in the litestream config","Watch for 429 throttling and back off on list-heavy follow loops"],"tags":["storage","s3","network","replication"],"backgroundTag":"api-request-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}