apache/hadoop · error · UnsupportedOperationException

This operation is not supported across two different storage

Error message

This operation is not supported across two different storage locations.

What it means

After both buckets exist, cross-bucket copy validation compares their metadata: if srcBucketInfo.getLocation() differs from dstBucketInfo.getLocation(), it throws UnsupportedOperationException("This operation is not supported across two different storage locations.") — the code block guards copies that would otherwise rely on the plain copy API, which cannot move data between regions (a rewrite would be required). A storage-class mismatch has a parallel check right below.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java:839

      // Avoid copy across locations or storage classes.
      if (!srcBucketName.equals(dstBucketName)) {
        StorageResourceId srcBucketResourceId = new StorageResourceId(srcBucketName);
        GoogleCloudStorageItemInfo srcBucketInfo =
            getGoogleCloudStorageItemInfo(gcsImpl, bucketInfoCache, srcBucketResourceId);
        if (!srcBucketInfo.exists()) {
          throw new FileNotFoundException("Bucket not found: " + srcBucketName);
        }

        StorageResourceId dstBucketResourceId = new StorageResourceId(dstBucketName);
        GoogleCloudStorageItemInfo dstBucketInfo =
            getGoogleCloudStorageItemInfo(gcsImpl, bucketInfoCache, dstBucketResourceId);
        if (!dstBucketInfo.exists()) {
          throw new FileNotFoundException("Bucket not found: " + dstBucketName);
        }

        // TODO: Restrict this only when copy-with-rewrite is enabled
        if (!srcBucketInfo.getLocation().equals(dstBucketInfo.getLocation())) {
          throw new UnsupportedOperationException(
              "This operation is not supported across two different storage locations.");
        }

        if (!srcBucketInfo.getStorageClass().equals(dstBucketInfo.getStorageClass())) {
          throw new UnsupportedOperationException(
              "This operation is not supported across two different storage classes.");
        }
      }
      checkArgument(
          !isNullOrEmpty(source.getObjectName()), "srcObjectName must not be null or empty");
      checkArgument(
          !isNullOrEmpty(destination.getObjectName()), "dstObjectName must not be null or empty");
      if (srcBucketName.equals(dstBucketName)
          && source.getObjectName().equals(destination.getObjectName())) {
        throw new IllegalArgumentException(
            String.format(
                "Copy destination must be different from source for %s.",
                StringPaths.fromComponents(srcBucketName, source.getObjectName())));

View on GitHub (pinned to 2add963021)

Solutions

1. Route cross-region transfers through a tool that uses the Rewrite API (gsutil cp/gsutil rsync, or storage.rewrite), which streams data across regions
2. Provision the destination bucket in the same location as the source and retry
3. For storage-class changes within the same bucket, use object storage-class updates instead of copy

Example fix

# before
hadoop distcp gs://src-us-bucket/data gs://dst-eu-bucket/out
# -> UnsupportedOperationException: across two different storage locations

# after (rewrite-based transfer handles cross-region)
gsutil -m cp -r gs://src-us-bucket/data gs://dst-eu-bucket/out

// Compare bucket locations before attempting cross-bucket copy
GoogleCloudStorageItemInfo srcInfo = gcs.getItemInfo(new StorageResourceId(srcBucket));
GoogleCloudStorageItemInfo dstInfo = gcs.getItemInfo(new StorageResourceId(dstBucket));
if (!srcInfo.getLocation().equals(dstInfo.getLocation())
    || !srcInfo.getStorageClass().equals(dstInfo.getStorageClass())) {
  throw new UnsupportedOperationException(
      "Use gsutil/rewrite for cross-location or cross-class transfers");
}
gcs.copy(map);

When it happens

Trigger: copy() with source and destination in different buckets that live in different GCS regions (e.g. us-central1 → europe-west1), or the analogous storage-class mismatch (e.g. STANDARD → COLDLINE) depending on the enabled code path.

Common situations: Cross-region data pipelines or DR copies configured with plain copy; mixing multi-region and regional buckets; promotion of data between environments hosted in different regions.

Related errors

• Error accessing Bucket %s
• This operation is not supported across two different buckets
• copy(%s->%s) failed.
• Bucket not found: %s
• This operation is not supported across two different storage

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6d9f9bc52ffc1d8c. Report an issue: GitHub.