apache/hadoop · error · ExitUtil.ExitException

49

49

Error message

Initialization failed because the bucket existence probefs.s3a.bucket.probe was not disabled. Check core-site settings.

What it means

To create a bucket that does not yet exist, the tool removes per-bucket overrides and sets fs.s3a.bucket.<bucket>.probe=0 before constructing the S3AFileSystem. If initialization still throws FileNotFoundException, the bucket existence probe ran anyway - fs.s3a.bucket.probe was not effectively disabled, most often because the property is declared <final>true</final> in core-site.xml so the runtime override cannot take effect. The tool converts this to exit code 49 (EXIT_BAD_CONFIGURATION) with PROBE_FAILURE; the printed message literally reads 'probefs.s3a.bucket.probe' because of a missing space in the string concatenation.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/tools/BucketTool.java:229

          LOG.info("{} = {}", key, v);
          return true;
        }).orElse(false);


    propagate.apply(AWS_REGION, region);
    propagate.apply(ENDPOINT, endpoint);

    // fail fast on third party store
    if (hasS3ExpressSuffix(bucket) && !isAwsEndpoint(endpoint.orElse(""))) {
      throw new ExitUtil.ExitException(EXIT_NOT_ACCEPTABLE, UNSUPPORTED_ZONE_ARG);
    }
    S3AFileSystem fs;
    try {
      fs = (S3AFileSystem) FileSystem.newInstance(fsURI, conf);
    } catch (FileNotFoundException e) {
      // this happens if somehow the probe wasn't disabled.
      errorln(PROBE_FAILURE);
      throw new ExitUtil.ExitException(EXIT_BAD_CONFIGURATION, PROBE_FAILURE);
    }

    try {

      // now build the configuration
      final CreateBucketConfiguration.Builder builder = CreateBucketConfiguration.builder();

      if (fs.hasPathCapability(new Path("/"), STORE_CAPABILITY_S3_EXPRESS_STORAGE)) {
        //  S3 Express store requires a zone and some other other settings
        final String az = zone.orElseThrow(() ->
            new ExitUtil.ExitException(EXIT_USAGE, NO_ZONE_SUPPLIED + bucket));
        builder.location(LocationInfo.builder()
                .type(LocationType.AVAILABILITY_ZONE).name(az).build())
            .bucket(software.amazon.awssdk.services.s3.model.BucketInfo.builder()
                .type(BucketType.DIRECTORY)
                .dataRedundancy(DataRedundancy.SINGLE_AVAILABILITY_ZONE).build());

      } else {

View on GitHub (pinned to 2add963021)

Solutions

1. Remove <final>true</final> from (or delete) fs.s3a.bucket.probe and fs.s3a.bucket.<bucket>.probe in core-site.xml and reload
2. Pre-create the bucket with the AWS CLI ('aws s3api create-bucket --bucket <name>') so the probe succeeds and the tool is not needed for creation
3. Grep all loaded config files for 'bucket.probe' and for '<final>' near S3A keys to find the locked property
4. Pass the intended configuration explicitly with -conf so the tool edits the same config it will run with

Example fix

<!-- before: core-site.xml blocks the override -->
<property>
  <name>fs.s3a.bucket.probe</name><value>2</value><final>true</final>
</property>
<!-- after: drop the final flag or the property entirely -->
<property>
  <name>fs.s3a.bucket.probe</name><value>0</value>
</property>

Configuration conf = new Configuration();
String global = conf.get("fs.s3a.bucket.probe");
String perBucket = conf.get("fs.s3a.bucket." + bucket + ".probe");
if (conf.getFinalParameters().contains(...)) { /* inspect final params for the probe keys */ }
// simplest preflight: ensure neither key is final before running the tool

try {
  int rc = new BucketTool(conf).exec(args);
} catch (ExitUtil.ExitException e) {
  if (e.getExitCode() == 49) {
    // EXIT_BAD_CONFIGURATION: fix core-site (remove final probe settings) or pre-create bucket
  }
}

When it happens

Trigger: core-site.xml declaring fs.s3a.bucket.probe or fs.s3a.bucket.<bucket>.probe with <final>true</final> while creating a brand-new bucket; configuration sources that re-assert the probe value after the tool's edits.

Common situations: Hardened enterprise clusters that mark S3A settings final; probe left at 2 (probe with any credential provider) in shared configs; ops teams unaware the create-bucket flow requires the probe disabled.

Related errors

• Failed to parse value %s as %s, property key %s
• No scheme in default FS: ${uri}
• Unsupported block buffer "{name}"
• Configuration hadoop.user.group.static.mapping.overrides is
• This committer does not work with the filesystem of type {sc

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9b029dfe437bd1bd. Report an issue: GitHub.