apache/hadoop · critical · FileNotFoundException

Bucket: %s not found.

Error message

Bucket: %s not found.

What it means

initialize() throws FileNotFoundException("Bucket: %s not found.") when ObjectStorageFactory.create(...).bucket() returns null, i.e. the bucket named in the tos:// URI authority could not be resolved/seen. The filesystem refuses to start so no operation silently targets the wrong place.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:614

  private void stopAllServices() {
    ThreadPools.shutdown(uploadThreadPool, 30, TimeUnit.SECONDS);
    ThreadPools.shutdown(taskThreadPool, 30, TimeUnit.SECONDS);
  }

  @Override
  public void initialize(URI name, Configuration conf) throws IOException {
    super.initialize(name, conf);
    setConf(conf);
    this.scheme = FSUtils.scheme(conf, name);

    // Username is the current user at the time the FS was instantiated.
    this.username = UserGroupInformation.getCurrentUser().getShortUserName();
    this.workingDir = new Path("/user", username).makeQualified(name, null);
    this.uri = URI.create(scheme + "://" + name.getAuthority());
    this.bucket = this.uri.getAuthority();
    this.storage = ObjectStorageFactory.create(scheme, bucket, getConf());
    if (storage.bucket() == null) {
      throw new FileNotFoundException(String.format("Bucket: %s not found.", name.getAuthority()));
    }

    int taskThreadPoolSize =
        getConf().getInt(ConfKeys.FS_TASK_THREAD_POOL_SIZE.key(storage.scheme()),
            ConfKeys.FS_TASK_THREAD_POOL_SIZE_DEFAULT);
    this.taskThreadPool = ThreadPools.newWorkerPool(TASK_THREAD_POOL_PREFIX, taskThreadPoolSize);

    int uploadThreadPoolSize =
        getConf().getInt(ConfKeys.FS_MULTIPART_THREAD_POOL_SIZE.key(storage.scheme()),
            ConfKeys.FS_MULTIPART_THREAD_POOL_SIZE_DEFAULT);
    this.uploadThreadPool =
        ThreadPools.newWorkerPool(MULTIPART_THREAD_POOL_PREFIX, uploadThreadPoolSize);

    if (storage.bucket().isDirectory()) {
      fsOps = new DirectoryFsOps((DirectoryStorage) storage, this::objectToFileStatus);
    } else {
      fsOps = new DefaultFsOps(storage, getConf(), taskThreadPool, this::objectToFileStatus);
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the bucket name spelling in the URI authority against the TOS console
2. Set the correct region endpoint (fs.tos.endpoint) for the bucket and retry
3. Validate credentials/permissions: the principal must be able to HeadBucket/ListBuckets on that bucket
4. Pre-flight in tests: assert the bucket is reachable before submitting jobs that assume it

Example fix

# before (core-site.xml)
<property><name>fs.tos.endpoint</name><value>tos-cn-beijing.volces.com</value></property>
# bucket actually lives in ap-southeast-1 -> initialize() fails with Bucket not found

# after
<property><name>fs.tos.endpoint</name><value>tos-ap-southeast-1.volces.com</value></property>

// pre-flight at job setup
Configuration c = new Configuration();
c.set("fs.tos.endpoint", expectedRegionEndpoint);
try (FileSystem fs = FileSystem.get(new URI("tos://" + bucket + "/"), c)) {
  fs.getFileStatus(new Path("/")); // forces initialize(); fails fast with Bucket not found
}

try { fs = FileSystem.get(tosUri, conf); }
catch (FileNotFoundException e) {
  if (e.getMessage().contains("Bucket:")) { /* check bucket name, endpoint, credentials */ }
  else throw e;
}

When it happens

Trigger: new Path('tos://my-buckt/path') typos in the authority; bucket in another region with the wrong fs.tos.endpoint configured; credentials whose grant does not permit ListBuckets/HeadBucket on that name; URI built from unescaped or empty authority.

Common situations: Region/endpoint mismatch after bucket migration; missing or wrong fs.tos.access-key/secret-key or credentials provider; environment-specific config (core-site.xml) pointing at a different account; test configs hitting a bucket deleted between runs.

Related errors

• Bucket {} does not exist
• Provider {this} has no credentials: {initializationException
• No COS Credential Providers
• Credentials requested after provider list was closed
• Bucket not found: %s

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/87a707a78a4f8956. Report an issue: GitHub.