{"record":{"id":"9b029dfe437bd1bd","repo":"apache/hadoop","slug":"49","errorCode":"49","errorMessage":"Initialization failed because the bucket existence probefs.s3a.bucket.probe was not disabled. Check core-site settings.","messagePattern":"Initialization failed because the bucket existence probefs\\.s3a\\.bucket\\.probe was not disabled\\. Check core-site settings\\.","errorType":"error_code","errorClass":"ExitUtil.ExitException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/tools/BucketTool.java","lineNumber":229,"sourceCode":"          LOG.info(\"{} = {}\", key, v);\n          return true;\n        }).orElse(false);\n\n\n    propagate.apply(AWS_REGION, region);\n    propagate.apply(ENDPOINT, endpoint);\n\n    // fail fast on third party store\n    if (hasS3ExpressSuffix(bucket) && !isAwsEndpoint(endpoint.orElse(\"\"))) {\n      throw new ExitUtil.ExitException(EXIT_NOT_ACCEPTABLE, UNSUPPORTED_ZONE_ARG);\n    }\n    S3AFileSystem fs;\n    try {\n      fs = (S3AFileSystem) FileSystem.newInstance(fsURI, conf);\n    } catch (FileNotFoundException e) {\n      // this happens if somehow the probe wasn't disabled.\n      errorln(PROBE_FAILURE);\n      throw new ExitUtil.ExitException(EXIT_BAD_CONFIGURATION, PROBE_FAILURE);\n    }\n\n    try {\n\n      // now build the configuration\n      final CreateBucketConfiguration.Builder builder = CreateBucketConfiguration.builder();\n\n      if (fs.hasPathCapability(new Path(\"/\"), STORE_CAPABILITY_S3_EXPRESS_STORAGE)) {\n        //  S3 Express store requires a zone and some other other settings\n        final String az = zone.orElseThrow(() ->\n            new ExitUtil.ExitException(EXIT_USAGE, NO_ZONE_SUPPLIED + bucket));\n        builder.location(LocationInfo.builder()\n                .type(LocationType.AVAILABILITY_ZONE).name(az).build())\n            .bucket(software.amazon.awssdk.services.s3.model.BucketInfo.builder()\n                .type(BucketType.DIRECTORY)\n                .dataRedundancy(DataRedundancy.SINGLE_AVAILABILITY_ZONE).build());\n\n      } else {","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/tools/BucketTool.java#L211-L247","documentation":"To create a bucket that does not yet exist, the tool removes per-bucket overrides and sets fs.s3a.bucket.<bucket>.probe=0 before constructing the S3AFileSystem. If initialization still throws FileNotFoundException, the bucket existence probe ran anyway - fs.s3a.bucket.probe was not effectively disabled, most often because the property is declared <final>true</final> in core-site.xml so the runtime override cannot take effect. The tool converts this to exit code 49 (EXIT_BAD_CONFIGURATION) with PROBE_FAILURE; the printed message literally reads 'probefs.s3a.bucket.probe' because of a missing space in the string concatenation.","triggerScenarios":"core-site.xml declaring fs.s3a.bucket.probe or fs.s3a.bucket.<bucket>.probe with <final>true</final> while creating a brand-new bucket; configuration sources that re-assert the probe value after the tool's edits.","commonSituations":"Hardened enterprise clusters that mark S3A settings final; probe left at 2 (probe with any credential provider) in shared configs; ops teams unaware the create-bucket flow requires the probe disabled.","solutions":["Remove <final>true</final> from (or delete) fs.s3a.bucket.probe and fs.s3a.bucket.<bucket>.probe in core-site.xml and reload","Pre-create the bucket with the AWS CLI ('aws s3api create-bucket --bucket <name>') so the probe succeeds and the tool is not needed for creation","Grep all loaded config files for 'bucket.probe' and for '<final>' near S3A keys to find the locked property","Pass the intended configuration explicitly with -conf so the tool edits the same config it will run with"],"exampleFix":"<!-- before: core-site.xml blocks the override -->\n<property>\n  <name>fs.s3a.bucket.probe</name><value>2</value><final>true</final>\n</property>\n<!-- after: drop the final flag or the property entirely -->\n<property>\n  <name>fs.s3a.bucket.probe</name><value>0</value>\n</property>","handlingStrategy":"validation","validationCode":"Configuration conf = new Configuration();\nString global = conf.get(\"fs.s3a.bucket.probe\");\nString perBucket = conf.get(\"fs.s3a.bucket.\" + bucket + \".probe\");\nif (conf.getFinalParameters().contains(...)) { /* inspect final params for the probe keys */ }\n// simplest preflight: ensure neither key is final before running the tool","typeGuard":null,"tryCatchPattern":"try {\n  int rc = new BucketTool(conf).exec(args);\n} catch (ExitUtil.ExitException e) {\n  if (e.getExitCode() == 49) {\n    // EXIT_BAD_CONFIGURATION: fix core-site (remove final probe settings) or pre-create bucket\n  }\n}","preventionTips":["Do not mark fs.s3a.bucket.probe or fs.s3a.bucket.<bucket>.probe as final in core-site.xml","Audit locked S3A properties before introducing bucket-creation automation","Pre-create buckets with the AWS CLI when config lockdown cannot be changed"],"tags":["s3a","configuration","core-site","bucket-probe","final-property","exit-code-49"],"backgroundTag":"misconfigured-filesystem","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}