{"record":{"id":"d4af03b9a29d0d4e","repo":"apache/beam","slug":"error-listing-object-keys-v","errorCode":null,"errorMessage":"error listing object keys: %v","messagePattern":"error listing object keys: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/io/filesystem/s3/s3.go","lineNumber":66,"sourceCode":"\tclient := s3.NewFromConfig(cfg)\n\treturn &fs{client: client}\n}\n\n// Close closes the filesystem.\nfunc (f *fs) Close() error {\n\treturn nil\n}\n\n// List returns a slice of the files in the filesystem that match the glob pattern.\nfunc (f *fs) List(ctx context.Context, glob string) ([]string, error) {\n\tbucket, keyPattern, err := parseURI(glob)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error parsing S3 uri: %v\", err)\n\t}\n\n\tkeys, err := f.listObjectKeys(ctx, bucket, keyPattern)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error listing object keys: %v\", err)\n\t}\n\n\tif len(keys) == 0 {\n\t\treturn nil, nil\n\t}\n\n\turis := make([]string, len(keys))\n\tfor i, key := range keys {\n\t\turis[i] = makeURI(bucket, key)\n\t}\n\n\treturn uris, nil\n}\n\n// listObjectKeys returns a slice of the keys in the bucket that match the key pattern.\nfunc (f *fs) listObjectKeys(\n\tctx context.Context,\n\tbucket string,","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/io/filesystem/s3/s3.go#L48-L84","documentation":"List() wraps any failure from listObjectKeys (which enumerates S3 object keys via ListObjectsV2) with this message. It means the S3 listing operation failed, e.g. due to credentials, bucket access, or API errors. The original error is embedded via %v so check the wrapped text for the root cause.","triggerScenarios":"Calling List(ctx, glob) on the S3 filesystem when the ListObjectsV2 call fails: missing/invalid AWS credentials, nonexistent bucket, insufficient s3:ListBucket permission, wrong region, or network failure.","commonSituations":"Default AWS credential chain not configured in the environment; IAM policy lacking ListBucket; bucket in a different region than the client config; transient network issues during long Beam pipelines.","solutions":["Fix the underlying error shown in the wrapped %v text (often an AWS SDK smithy operation error).","Verify AWS credentials are available (env vars, ~/.aws/credentials, or instance role).","Confirm the bucket exists and the client region matches the bucket region.","Grant the caller IAM permission s3:ListBucket on the bucket."],"exampleFix":"// before\nclient := s3.NewFromConfig(cfg) // cfg.Region may be empty\n// after\nawsCfg, _ := config.LoadDefaultConfig(ctx, config.WithRegion(\"us-east-1\"))\nclient := s3.NewFromConfig(awsCfg)","handlingStrategy":"retry","validationCode":"// ensure credentials & bucket reachable before List\nsess := credentials.NewEnvCredentials()\nc, err := sess.Retrieve(ctx)\nif err != nil { return fmt.Errorf(\"no AWS credentials: %w\", err) }\nif _, err := f.client.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: aws.String(bucket)}); err != nil {\n  return fmt.Errorf(\"bucket %s unreachable: %w\", bucket, err)\n}","typeGuard":null,"tryCatchPattern":"keys, err := fs.List(ctx, glob)\nif err != nil {\n  if isRetryable(err) { // e.g. throttling / network\n     return retryWithBackoff(ctx, func() error { _, err = fs.List(ctx, glob); return err })\n  }\n  return fmt.Errorf(\"s3 list failed for %s: %w\", glob, err)\n}","preventionTips":["Configure AWS credentials via a standard chain before running pipelines.","Verify s3:ListBucket IAM permission for the target bucket.","Match client region to bucket region.","Enable SDK retryer with backoff for throttling."],"tags":["aws","s3","network","io"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}