apache/hadoop · error · UnsupportedOperationException

This operation is not supported across two different storage

Error message

This operation is not supported across two different storage classes.

What it means

Thrown by GoogleCloudStorage.copy when the source and destination buckets have different GCS storage classes (STANDARD, NEARLINE, COLDLINE, ARCHIVE). The connector fetches both buckets' metadata (via a bucketInfoCache) and refuses the copy before it starts; a source comment notes the check is broader than necessary and should apply only when copy-with-rewrite is enabled. An equivalent check for differing bucket locations throws just above it.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java:844

        if (!srcBucketInfo.exists()) {
          throw new FileNotFoundException("Bucket not found: " + srcBucketName);
        }

        StorageResourceId dstBucketResourceId = new StorageResourceId(dstBucketName);
        GoogleCloudStorageItemInfo dstBucketInfo =
            getGoogleCloudStorageItemInfo(gcsImpl, bucketInfoCache, dstBucketResourceId);
        if (!dstBucketInfo.exists()) {
          throw new FileNotFoundException("Bucket not found: " + dstBucketName);
        }

        // TODO: Restrict this only when copy-with-rewrite is enabled
        if (!srcBucketInfo.getLocation().equals(dstBucketInfo.getLocation())) {
          throw new UnsupportedOperationException(
              "This operation is not supported across two different storage locations.");
        }

        if (!srcBucketInfo.getStorageClass().equals(dstBucketInfo.getStorageClass())) {
          throw new UnsupportedOperationException(
              "This operation is not supported across two different storage classes.");
        }
      }
      checkArgument(
          !isNullOrEmpty(source.getObjectName()), "srcObjectName must not be null or empty");
      checkArgument(
          !isNullOrEmpty(destination.getObjectName()), "dstObjectName must not be null or empty");
      if (srcBucketName.equals(dstBucketName)
          && source.getObjectName().equals(destination.getObjectName())) {
        throw new IllegalArgumentException(
            String.format(
                "Copy destination must be different from source for %s.",
                StringPaths.fromComponents(srcBucketName, source.getObjectName())));
      }
    }
  }

  private static GoogleCloudStorageItemInfo getGoogleCloudStorageItemInfo(

View on GitHub (pinned to 2add963021)

Solutions

Make source and destination bucket storage classes match (recreate or transition the destination bucket, or pick a same-class destination).
Keep the copy within one bucket and change the object's storage class with a lifecycle rule or `gcloud storage objects update` / gsutil rewrite, which use the Rewrite API that handles class changes.
Use Storage Transfer Service or `gcloud storage cp` for the cross-class transfer instead of the connector's copy API.
If you maintain connector code, narrow the check to copy-with-rewrite mode (per the TODO) only after testing rewrite behavior across classes.

Example fix

// before
gcs.copy(
    new StorageResourceId("hot-bkt", "data/file1"),
    new StorageResourceId("archive-bkt", "data/file1")); // throws: class mismatch

// after: same-class destination bucket, or rewrite in place
// gcloud storage objects update gs://hot-bkt/data/file1 --storage-class=COLDLINE
// (or) gsutil -s COLDLINE cp -r gs://hot-bkt gs://archive-bkt

Defensive patterns

Strategy: validation

Validate before calling

// Compare both bucket metadata before copying
GoogleCloudStorageItemInfo srcB = gcs.getItemInfo(new StorageResourceId(srcBucket));
GoogleCloudStorageItemInfo dstB = gcs.getItemInfo(new StorageResourceId(dstBucket));
checkState(srcB.exists() && dstB.exists(), "both buckets must exist");
checkState(Objects.equals(srcB.getLocation(), dstB.getLocation()), "location mismatch");
checkState(Objects.equals(srcB.getStorageClass(), dstB.getStorageClass()), "storage class mismatch");
gcs.copy(srcId, dstId);

Try / catch

try { gcs.copy(srcId, dstId); } catch (UnsupportedOperationException e) { if (e.getMessage().contains("storage classes") || e.getMessage().contains("storage locations")) { /* fall back to gcloud rewrite / transfer, or same-class bucket */ } else throw e; }

Prevention

Standardize storage class and location for buckets used as copy/rename sources and destinations.
Encode the location/class constraint in bucket-creation templates so new buckets match.
For archival to cold classes, plan lifecycle rules or rewrite-based transfer instead of connector copy.

When it happens

Trigger: Calling gcs.copy(source, destination) where srcBucketInfo.getStorageClass() != dstBucketInfo.getStorageClass() — e.g. STANDARD -> COLDLINE/ARCHIVE. Both buckets must exist (a missing bucket throws FileNotFoundException earlier), and src/dst object names must be non-empty or checkArgument fires first.

Common situations: Archival pipelines copying hot data into cheaper-class buckets; cross-project backups where the destination bucket was created with a different default storage class; bucket class migrations; distcp or GHFS rename workflows that route through GoogleCloudStorage.copy across buckets.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/941a8fadc5b263f0. Report an issue: GitHub.