apache/hadoop · error · IllegalArgumentException

Wrong bucket: %s, in path: %s, expected bucket: %s

Error message

Wrong bucket: %s, in path: %s, expected bucket: %s

What it means

checkPath also validates the bucket: a Path's authority must be null (bucket-less, qualified later) or equal the root bucket of this FileSystem instance (fsRoot's authority), otherwise IllegalArgumentException 'Wrong bucket'. One GoogleHadoopFileSystem instance serves exactly one GCS bucket.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopFileSystem.java:237

    // Validate scheme
    URI uri = path.toUri();

    String scheme = uri.getScheme();
    if (scheme != null && !scheme.equalsIgnoreCase(getScheme())) {
      throw new IllegalArgumentException(
          String.format("Wrong scheme: %s, in path: %s, expected scheme: %s", scheme, path,
              getScheme()));
    }

    String bucket = uri.getAuthority();
    String rootBucket = fsRoot.toUri().getAuthority();

    // Bucket-less URIs will be qualified later
    if (bucket == null || bucket.equals(rootBucket)) {
      return;
    }

    throw new IllegalArgumentException(
        String.format("Wrong bucket: %s, in path: %s, expected bucket: %s", bucket, path,
            rootBucket));
  }

  /**
   * Validates that GCS path belongs to this file system. The bucket must match the root bucket
   * provided at initialization time.
   */
  Path getHadoopPath(final URI gcsPath) {
    LOG.trace("getHadoopPath(gcsPath: {})", gcsPath);

    // Handle root. Delegate to getGcsPath on "gs:/" to resolve the appropriate gs://<bucket> URI.
    if (gcsPath.equals(getGcsPath(fsRoot))) {
      return fsRoot;
    }

    StorageResourceId resourceId = StorageResourceId.fromUriPath(gcsPath, true);

View on GitHub (pinned to 2add963021)

Solutions

1. Acquire a separate FileSystem per bucket: FileSystem.get(new URI("gs://bucket-b"), conf) or path.getFileSystem(conf).
2. Use bucket-less (relative) paths so they qualify against the instance's root bucket.
3. Align job path configs with the bucket the FileSystem was initialized for (or configure per-bucket roots).

Example fix

// before
FileSystem fsA = FileSystem.get(new URI("gs://bucket-a"), conf);
fsA.exists(new Path("gs://bucket-b/key")); // IllegalArgumentException: Wrong bucket

// after
FileSystem fsB = FileSystem.get(new URI("gs://bucket-b"), conf);
fsB.exists(new Path("gs://bucket-b/key"));

static boolean bucketOk(URI fsRoot, Path p) {
  String b = p.toUri().getAuthority();
  return b == null || b.equals(fsRoot.getAuthority());
}

catch IllegalArgumentException with message startsWith("Wrong bucket") - acquire a FileSystem rooted at that bucket (FileSystem.get(new URI("gs://" + bucket), conf)) and retry.

When it happens

Trigger: FileSystem.get(new URI("gs://bucket-a"), conf) then operations on gs://bucket-b/... paths; paths built with a different bucket authority than the instance's root.

Common situations: Multi-bucket pipelines (raw/staging/processed buckets) reusing one FileSystem handle; fs.defaultFS bound to one bucket while job configs reference others; templates that swap buckets in path strings but not the FileSystem URI.

Related errors

• Wrong scheme: %s, in path: %s, expected scheme: %s
• Rename dir to self is forbidden
• GCS path supports only '%s' scheme, instead got '%s' from '%
• GCS bucket name cannot be empty.
• Invalid GCS bucket name '%s': bucket name must contain only

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b385de7ccbb4d400. Report an issue: GitHub.