{"record":{"id":"5c109c1e97992080","repo":"apache/beam","slug":"error-retrieving-page-v","errorCode":null,"errorMessage":"error retrieving page: %v","messagePattern":"error retrieving page: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/io/filesystem/s3/s3.go","lineNumber":98,"sourceCode":"\n// listObjectKeys returns a slice of the keys in the bucket that match the key pattern.\nfunc (f *fs) listObjectKeys(\n\tctx context.Context,\n\tbucket string,\n\tkeyPattern string,\n) ([]string, error) {\n\tprefix := fsx.GetPrefix(keyPattern)\n\tparams := &s3.ListObjectsV2Input{\n\t\tBucket: aws.String(bucket),\n\t\tPrefix: aws.String(prefix),\n\t}\n\tpaginator := s3.NewListObjectsV2Paginator(f.client, params)\n\n\tvar objects []string\n\tfor paginator.HasMorePages() {\n\t\toutput, err := paginator.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error retrieving page: %v\", err)\n\t\t}\n\n\t\tfor _, object := range output.Contents {\n\t\t\tkey := aws.ToString(object.Key)\n\t\t\tmatch, err := filepath.Match(keyPattern, key)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"invalid key pattern: %s\", keyPattern)\n\t\t\t}\n\n\t\t\tif match {\n\t\t\t\tobjects = append(objects, key)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn objects, nil\n}\n","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/io/filesystem/s3/s3.go#L80-L116","documentation":"listObjectKeys fetches S3 object listings page by page with ListObjectsV2Paginator; this error wraps a failure to retrieve any single page. It indicates the S3 API rejected or failed on a pagination request, typically auth, permission, throttling, or network problems. It is then further wrapped by List's 'error listing object keys' message.","triggerScenarios":"paginator.NextPage(ctx) returns an error while iterating HasMorePages() — e.g. AccessDenied on ListObjectsV2, slow-down throttling on large buckets, expired session credentials mid-iteration, or network interruption.","commonSituations":"Large buckets requiring many pages hit S3 throttling (SlowDown); temporary credentials (STS) expire during iteration; IAM role not yet propagated in a fresh environment.","solutions":["Inspect the wrapped AWS error; handle smithy APIError codes like AccessDenied or SlowDown.","Retry with backoff on throttling; consider retryer options on the S3 client config.","Refresh or fix credentials if session tokens expired.","Check s3:ListBucket permission for the exact bucket."],"exampleFix":"// before\nclient := s3.NewFromConfig(cfg)\n// after\nclient := s3.NewFromConfig(cfg, func(o *s3.Options) {\n  o.Retryer = retry.AddWithMaxBackoffDelay(retry.NewStandard(), 5*time.Second)\n})","handlingStrategy":"retry","validationCode":"// check permissions/creds before paginating\ncreds, err := awsConfig.Credentials.Retrieve(ctx)\nif err != nil { return err }\n// probe with a single small listing\np := s3.NewListObjectsV2Paginator(f.client, &s3.ListObjectsV2Input{Bucket: aws.String(bucket), MaxKeys: aws.Int32(1)})\nif _, err := p.NextPage(ctx); err != nil { return err }","typeGuard":null,"tryCatchPattern":"_, err := paginator.NextPage(ctx)\nif err != nil {\n  var re *retry.MaxAttemptsError\n  if errors.As(err, &re) || isThrottle(err) {\n    time.Sleep(backoff); continue\n  }\n  return fmt.Errorf(\"page fetch failed: %w\", err)\n}","preventionTips":["Use a standard retryer with exponential backoff on the S3 client.","Refresh STS/temporary credentials before long iterations.","Probe with MaxKeys=1 to validate access before large scans.","Watch for SlowDown errors on very large buckets and shard prefixes."],"tags":["aws","s3","pagination","network"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}