What causes "Moving object from bucket '%s' to '%s' is not supported" in apache/hadoop?

Replace the rename with copy + delete: GoogleCloudStorage.copy (requires same location and storage class) followed by source deletion.

apache/hadoop · error · UnsupportedOperationException

Moving object from bucket '%s' to '%s' is not supported

Error message

Moving object from bucket '%s' to '%s' is not supported

What it means

rename only supports moves within a single bucket. After the same-bucket branches (including bucket-root handling) fall through, differing src/dst buckets produce UnsupportedOperationException — the source carries a TODO to add across-bucket moves. Cross-bucket transfer at the GoogleCloudStorage.copy layer additionally requires equal bucket location and storage class.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageFileSystem.java:500

            dst, /* allowEmptyObjectName= */ true, /* generationId= */ 0L);
    if (srcResourceId.getBucketName().equals(dstResourceId.getBucketName())) {
      // First, move all items except marker items
      moveInternal(srcToDstItemNames);
      // Finally, move marker items (if any) to mark rename operation success
      moveInternal(srcToDstMarkerItemNames);

      if (srcInfo.getItemInfo().isBucket()) {
        deleteBucket(Collections.singletonList(srcInfo));
      } else {
        // If src is a directory then srcItemInfos does not contain its own name,
        // we delete item separately in the list.
        deleteObjects(Collections.singletonList(srcInfo));
      }
      return;
    }

    // TODO: Add support for across bucket moves
    throw new UnsupportedOperationException(String.format(
        "Moving object from bucket '%s' to '%s' is not supported",
        srcResourceId.getBucketName(),
        dstResourceId.getBucketName()));
  }

  List<FileInfo> listFileInfoForPrefix(URI prefix, ListFileOptions listOptions)
      throws IOException {
    LOG.trace("listAllFileInfoForPrefix(prefix: {})", prefix);
    StorageResourceId prefixId = getPrefixId(prefix);
    List<GoogleCloudStorageItemInfo> itemInfos =
        gcs.listDirectoryRecursive(prefixId.getBucketName(), prefixId.getObjectName());
    List<FileInfo> fileInfos = FileInfo.fromItemInfos(itemInfos);
    fileInfos.sort(FILE_INFO_PATH_COMPARATOR);
    return fileInfos;
  }

  /** Moves items in given map that maps source items to destination items. */
  private void moveInternal(Map<FileInfo, URI> srcToDstItemNames) throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

Replace the rename with copy + delete: GoogleCloudStorage.copy (requires same location and storage class) followed by source deletion.
For bulk data use DistCp (update/delete semantics) or Storage Transfer Service instead of per-file rename.
Restructure the job to write directly into the destination bucket so no move is needed.

Example fix

// before
fs.rename(URI.create("gs://a/f"), URI.create("gs://b/f")); // UnsupportedOperationException

// after: copy + delete within connector limits
gcs.copy(new StorageResourceId("a", "f"), new StorageResourceId("b", "f"));
gcs.deleteObjects(List.of(new StorageResourceId("a", "f")));

Defensive patterns

Strategy: fallback

Validate before calling

if (!src.getBucket().equals(dst.getBucket())) {
  // connector cannot rename across buckets: copy + delete instead
  gcs.copy(new StorageResourceId(src.getBucket(), src.getObject()),
           new StorageResourceId(dst.getBucket(), dst.getObject()));
  gcs.deleteObjects(List.of(new StorageResourceId(src.getBucket(), src.getObject())));
} else {
  fs.rename(src, dst);
}

Try / catch

catch (UnsupportedOperationException e) {
  if (e.getMessage().contains("not supported")) { /* switch to copy+delete or distcp */ } else throw e; }

Prevention

Never rely on rename for cross-bucket moves in GCS-backed filesystems.
Keep copy source/destination buckets matched on location and storage class so the fallback copy works.
For large moves, plan DistCp or Storage Transfer Service rather than per-file copy+delete.

When it happens

Trigger: GHFS/GoogleCloudStorageFileSystem rename from gs://bucket-a/... to gs://bucket-b/... for any non-bucket-root item — e.g. a job 'moving' outputs to another bucket via rename.

Common situations: Pipelines ported from HDFS where cross-directory rename is free; multi-bucket layouts expecting transparent moves; staging-to-publish workflows across buckets.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6a01252304ea9443. Report an issue: GitHub.