{"record":{"id":"2a6d195cda198424","repo":"apache/hadoop","slug":"bucket-does-not-exist-accessing-with-fs-s3a-endp","errorCode":null,"errorMessage":" Bucket does not exist. Accessing with fs.s3a.endpoint set to <endpoint>","messagePattern":" Bucket does not exist\\. Accessing with fs\\.s3a\\.endpoint set to <endpoint>","errorType":"exception","errorClass":"UnknownStoreException","httpStatus":404,"severity":"critical","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":1062,"sourceCode":"\n    if(!trackDurationAndSpan(\n        STORE_EXISTS_PROBE, bucket, null, () ->\n            invoker.retry(\"doesBucketExist\", bucket, true, () -> {\n              try {\n                getS3Client().headBucket(HeadBucketRequest.builder().bucket(bucket).build());\n                return true;\n              } catch (AwsServiceException ex) {\n                int statusCode = ex.statusCode();\n                if (statusCode == SC_404_NOT_FOUND ||\n                    (statusCode == SC_403_FORBIDDEN && accessPoint != null)) {\n                  return false;\n                }\n              }\n\n              return true;\n            }))) {\n\n      throw new UnknownStoreException(\"s3a://\" + bucket + \"/\",\n          \" Bucket does \" + \"not exist. \" + \"Accessing with \" + ENDPOINT + \" set to \"\n              + getConf().getTrimmed(ENDPOINT, null));\n    }\n  }\n\n  /**\n   * Get S3A Instrumentation. For test purposes.\n   * @return this instance's instrumentation.\n   */\n  @VisibleForTesting\n  public S3AInstrumentation getInstrumentation() {\n    return instrumentation;\n  }\n\n  /**\n   * Get FS Statistic for this S3AFS instance.\n   *\n   * @return FS statistic instance.","sourceCodeStart":1044,"sourceCodeEnd":1080,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L1044-L1080","documentation":"Thrown by S3AFileSystem.verifyBucketExists() during initialize(): a headBucket probe returned HTTP 404, or 403 when the bucket name is an S3 access point, so no bucket with that name is visible at the configured endpoint. The FileSystem cannot be constructed and every later operation is blocked. The message deliberately includes the s3a://bucket/ URI and the fs.s3a.endpoint value to make endpoint/bucket mismatches diagnosable.","triggerScenarios":"FileSystem.get() or any path qualification for s3a://<bucket>/ where the bucket does not exist at the endpoint reached by the client; fs.s3a.endpoint pointing at the wrong region or at a third-party store (MinIO, Ceph) where the bucket was never created; using an access point ARN whose 403 is interpreted as 'not found'.","commonSituations":"Typo in the bucket name in a job URL; bucket deleted or owned by another account; fs.s3a.endpoint/fs.s3a.endpoint.region left stale after a region migration; integration tests against a containerized S3 that was restarted without recreating buckets.","solutions":["Verify and fix the bucket name, and create the bucket if missing (aws s3api head-bucket / aws s3 mb s3://<bucket>)","Set fs.s3a.endpoint.region (and fs.s3a.endpoint for non-AWS stores) to the region that actually hosts the bucket","For access points, confirm the ARN and that the credentials have permission on it - a 403 is treated as 'bucket absent'","Check credentials and endpoint routing: a mis-signed request can surface as a missing bucket"],"exampleFix":"<!-- before: bucket is in eu-west-1, client signs for us-east-1 -->\n<property><name>fs.s3a.endpoint.region</name><value>us-east-1</value></property>\n\n<!-- after -->\n<property><name>fs.s3a.endpoint.region</name><value>eu-west-1</value></property>","handlingStrategy":"try-catch","validationCode":"// Preflight before handing the FileSystem to a long job\ntry {\n  FileSystem fs = FileSystem.get(new URI(\"s3a://\" + bucket + \"/\"), conf);\n} catch (UnknownStoreException e) {\n  throw new IOException(\"Bucket \" + bucket + \" absent or wrong endpoint: \" + e.getMessage(), e);\n}","typeGuard":"static boolean isUnknownStore(Throwable t) {\n  return t instanceof org.apache.hadoop.fs.UnknownStoreException;\n}","tryCatchPattern":"When creating the FileSystem, catch UnknownStoreException separately and before IOException: it is a deterministic configuration error - fail fast with the bucket/endpoint in the message; do not retry it.","preventionTips":["Provision and verify buckets (aws s3api head-bucket) before jobs run","Pin fs.s3a.endpoint.region to the bucket's region in core-site.xml","Validate bucket names from user input before building s3a:// URIs","In CI, assert the store endpoint is reachable before the suite starts"],"tags":["s3a","s3","bucket","unknown-store","initialization","configuration"],"backgroundTag":"s3-bucket-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}