{"record":{"id":"d6d2aad22dbf78a8","repo":"JuliusBrussee/caveman","slug":"objectstore-list-q-w","errorCode":null,"errorMessage":"objectstore: list %q: %w","messagePattern":"objectstore: list %q: %w","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"shared/platform/objectstore/objectstore.go","lineNumber":360,"sourceCode":"\treturn nil\n}\n\nfunc (m *minioStore) Exists(ctx context.Context, key string) (bool, error) {\n\t_, err := m.client.StatObject(ctx, m.bucket, key, minio.StatObjectOptions{})\n\tif err != nil {\n\t\tif minio.ToErrorResponse(err).Code == \"NoSuchKey\" {\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, fmt.Errorf(\"objectstore: stat %q: %w\", key, err)\n\t}\n\treturn true, nil\n}\n\nfunc (m *minioStore) List(ctx context.Context, prefix string) ([]string, error) {\n\tvar keys []string\n\tfor obj := range m.client.ListObjects(ctx, m.bucket, minio.ListObjectsOptions{Prefix: prefix, Recursive: true}) {\n\t\tif obj.Err != nil {\n\t\t\treturn nil, fmt.Errorf(\"objectstore: list %q: %w\", prefix, obj.Err)\n\t\t}\n\t\tkeys = append(keys, obj.Key)\n\t}\n\treturn keys, nil\n}\n\nfunc (m *minioStore) ListPage(ctx context.Context, prefix, startAfter string, maxKeys int) (ObjectPage, error) {\n\tif maxKeys <= 0 {\n\t\treturn ObjectPage{}, fmt.Errorf(\"objectstore: page size must be positive\")\n\t}\n\t// Request one extra key so the caller can tell whether another page exists,\n\t// while cancelling the producer as soon as the bounded result is known.\n\tpageCtx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\tkeys := make([]string, 0, maxKeys)\n\tvar extra string\n\tfor obj := range m.client.ListObjects(pageCtx, m.bucket, minio.ListObjectsOptions{\n\t\tPrefix: prefix, Recursive: true, StartAfter: startAfter, MaxKeys: maxKeys + 1,","sourceCodeStart":342,"sourceCodeEnd":378,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/shared/platform/objectstore/objectstore.go#L342-L378","documentation":"minioStore.List wraps obj.Err produced while iterating ListObjects with the given prefix (recursive). The listing stream is lazy; errors arrive mid-iteration as per-entry errors, so a partially built keys slice is discarded and the prefix plus cause are wrapped with %w.","triggerScenarios":"Calling List(ctx, prefix) when the recursive listing fails partway: AccessDenied on s3:ListBucket, connection drop during pagination, SlowDown throttling, or invalid prefix causing backend errors.","commonSituations":"Large prefixes that require many paginated LIST calls over unstable networks; IAM missing list permission; heavy concurrent listing triggering rate limits; S3-compatible backends with pagination quirks.","solutions":["Unwrap and check minio.ErrorResponse for AccessDenied (grant s3:ListBucket) or SlowDown (backoff).","Retry the whole List on transient errors; results are recomputed from scratch.","Use ListPage with bounded maxKeys to reduce per-call work on huge prefixes.","Check network stability/timeout configuration for long listings.","Avoid extremely broad prefixes; narrow them to reduce pagination."],"exampleFix":"// before\nkeys, err := store.List(ctx, \"\") // lists entire bucket\n// after\nkeys, err := store.List(ctx, \"uploads/2026/09/\") // narrow prefix, fewer pages\nif err != nil {\n    return fmt.Errorf(\"list uploads: %w\", err)\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"keys, err := store.List(ctx, prefix)\nif err != nil {\n    var mr minio.ErrorResponse\n    if errors.As(err, &mr) && mr.Code == \"SlowDown\" {\n        time.Sleep(backoff.Next())\n        keys, err = store.List(ctx, prefix)\n    }\n    if err != nil { return err }\n}","preventionTips":["Use narrow, structured prefixes instead of listing whole buckets.","Throttle concurrent List calls to avoid SlowDown.","Prefer ListPage for large prefixes to bound each request.","Grant s3:ListBucket to service accounts that enumerate objects."],"tags":["object-storage","list","minio","pagination"],"backgroundTag":"database-query-failed","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}