apache/hadoop · error · UnsupportedOperationException

This operation is not supported across two different buckets

Error message

This operation is not supported across two different buckets.

What it means

The move (rename) validation loop rejects any source/destination pair whose bucket names differ, throwing UnsupportedOperationException("This operation is not supported across two different buckets."). The connector implements move as a same-bucket metadata operation (copy + delete); a cross-bucket move would require cross-bucket rewriting, which it deliberately does not attempt.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java:721

   * Preconditions} in addition to checking for src/dst bucket equality.
   */
  static void validateMoveArguments(
      Map<StorageResourceId, StorageResourceId> sourceToDestinationObjectsMap) throws IOException {
    checkNotNull(sourceToDestinationObjectsMap, "srcObjects must not be null");

    if (sourceToDestinationObjectsMap.isEmpty()) {
      return;
    }

    for (Map.Entry<StorageResourceId, StorageResourceId> entry :
        sourceToDestinationObjectsMap.entrySet()) {
      StorageResourceId source = entry.getKey();
      StorageResourceId destination = entry.getValue();
      String srcBucketName = source.getBucketName();
      String dstBucketName = destination.getBucketName();
      // Avoid move across buckets.
      if (!srcBucketName.equals(dstBucketName)) {
        throw new UnsupportedOperationException(
            "This operation is not supported across two different buckets.");
      }
      checkArgument(
          !isNullOrEmpty(source.getObjectName()), "srcObjectName must not be null or empty");
      checkArgument(
          !isNullOrEmpty(destination.getObjectName()), "dstObjectName must not be null or empty");
      if (srcBucketName.equals(dstBucketName)
          && source.getObjectName().equals(destination.getObjectName())) {
        throw new IllegalArgumentException(
            String.format(
                "Move destination must be different from source for %s.",
                StringPaths.fromComponents(srcBucketName, source.getObjectName())));
      }
    }
  }

  void copy(Map<StorageResourceId, StorageResourceId> sourceToDestinationObjectsMap)
      throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Perform copy-then-delete explicitly for cross-bucket moves (gcs.copy supports cross-bucket under conditions)
2. Keep rename source and destination inside the same bucket
3. Restructure the pipeline so intermediate data lives in the destination bucket

Example fix

// before
fs.rename(new Path("gs://bucketA/src"), new Path("gs://bucketB/dst"));
// -> UnsupportedOperationException: across two different buckets

// after
FileSystem srcFs = ...; // bucketA
srcFs.rename(new Path("gs://bucketA/src"), new Path("gs://bucketA/dst")); // intra-bucket only
// for cross-bucket: FileUtil.copy(srcFs, srcPath, dstFs, dstPath, true, conf);

// Guard renames: same bucket only
if (!srcPath.toUri().getHost().equals(dstPath.toUri().getHost())) {
  throw new UnsupportedOperationException("cross-bucket rename not allowed; use copy+delete");
}
fs.rename(srcPath, dstPath);

When it happens

Trigger: fs.rename() (or GoogleCloudStorage move API) with src and dst on different buckets — e.g. renaming gs://bucketA/dir into gs://bucketB/dir. Fails during argument validation before any service call.

Common situations: Distcp or job config placing work and output dirs on different buckets and then renaming between them; scripts assuming POSIX rename works across mounts; porting code from another connector that allowed cross-bucket rename.

Related errors

• Moving object from bucket '%s' to '%s' is not supported
• Bucket doesn't match for source '%s' and destination '%s'!
• Move destination must be different from source for %s.
• This operation is not supported across two different storage
• This operation is not supported across two different storage

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bf480b97251b08ef. Report an issue: GitHub.