{"record":{"id":"d5efc88c3133dc18","repo":"JuliusBrussee/caveman","slug":"objectstore-list-page-q-w","errorCode":null,"errorMessage":"objectstore: list page %q: %w","messagePattern":"objectstore: list page %q: %w","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"shared/platform/objectstore/objectstore.go","lineNumber":381,"sourceCode":"\t}\n\treturn keys, nil\n}\n\nfunc (m *minioStore) ListPage(ctx context.Context, prefix, startAfter string, maxKeys int) (ObjectPage, error) {\n\tif maxKeys <= 0 {\n\t\treturn ObjectPage{}, fmt.Errorf(\"objectstore: page size must be positive\")\n\t}\n\t// Request one extra key so the caller can tell whether another page exists,\n\t// while cancelling the producer as soon as the bounded result is known.\n\tpageCtx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\tkeys := make([]string, 0, maxKeys)\n\tvar extra string\n\tfor obj := range m.client.ListObjects(pageCtx, m.bucket, minio.ListObjectsOptions{\n\t\tPrefix: prefix, Recursive: true, StartAfter: startAfter, MaxKeys: maxKeys + 1,\n\t}) {\n\t\tif obj.Err != nil {\n\t\t\treturn ObjectPage{}, fmt.Errorf(\"objectstore: list page %q: %w\", prefix, obj.Err)\n\t\t}\n\t\tif len(keys) < maxKeys {\n\t\t\tkeys = append(keys, obj.Key)\n\t\t} else {\n\t\t\textra = obj.Key\n\t\t\tbreak\n\t\t}\n\t}\n\tif extra == \"\" {\n\t\treturn ObjectPage{Keys: keys}, nil\n\t}\n\treturn ObjectPage{Keys: keys, NextToken: keys[len(keys)-1], Truncated: true}, nil\n}\n","sourceCodeStart":363,"sourceCodeEnd":395,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/shared/platform/objectstore/objectstore.go#L363-L395","documentation":"ListPage fetches maxKeys+1 keys via ListObjects (StartAfter, MaxKeys) to detect an extra page; this error wraps obj.Err from that listing stream. As with List, listing errors surface lazily during iteration, aborting the page. The prefix and cause are preserved with %w.","triggerScenarios":"Calling ListPage(ctx, prefix, startAfter, maxKeys) when the bounded listing fails: AccessDenied on s3:ListBucket, throttling (SlowDown), connection drop mid-page, or a backend that mishandles MaxKeys/StartAfter combinations.","commonSituations":"StartAfter values that don't match the prefix's key layout (some backends error or misbehave); tight pagination loops hitting rate limits; S3-compatible stores with MaxKeys caps lower than requested.","solutions":["Unwrap and check minio.ErrorResponse for AccessDenied (grant s3:ListBucket) or SlowDown (add backoff between pages).","Retry the page request on transient failures — pagination is stateless thanks to startAfter.","Verify startAfter is a valid key within the prefix ordering.","Reduce maxKeys if the backend caps MaxKeys and errors on larger values.","Check for infinite pagination loops (startAfter not advancing) that exhaust rate limits."],"exampleFix":"// before\nfor {\n    page, err := store.ListPage(ctx, prefix, cursor, 1000)\n    if err != nil { return err }\n    cursor = page.Next\n}\n// after\nfor {\n    page, err := store.ListPage(ctx, prefix, cursor, 1000)\n    if err != nil {\n        return backoff.Retry(ctx, func() error {\n            _, rerr := store.ListPage(ctx, prefix, cursor, 1000)\n            return rerr\n        })\n    }\n    if page.Next == \"\" || page.Next == cursor { break }\n    cursor = page.Next\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"page, err := store.ListPage(ctx, prefix, startAfter, maxKeys)\nif err != nil {\n    var mr minio.ErrorResponse\n    if errors.As(err, &mr) && (mr.Code == \"SlowDown\" || mr.Code == \"RequestTimeout\") {\n        return retryWithBackoff(ctx, func() error {\n            page, err = store.ListPage(ctx, prefix, startAfter, maxKeys)\n            return err\n        })\n    }\n    return err\n}","preventionTips":["Advance the startAfter cursor every page to avoid loop-induced throttling.","Use page sizes within backend MaxKeys limits.","Add jittered backoff between pagination rounds.","Keep startAfter keys in the same lexical order the listing returns."],"tags":["object-storage","list","pagination","minio"],"backgroundTag":"database-query-failed","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}