{"record":{"id":"ebb406320ed57c54","repo":"apache/hadoop","slug":"bucket-doesn-t-match-for-source-s-and-destinati","errorCode":null,"errorMessage":"Bucket doesn't match for source '%s' and destination '%s'!","messagePattern":"Bucket doesn't match for source '(.+?)' and destination '(.+?)'!","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java","lineNumber":373,"sourceCode":"      } else {\n        if (ErrorTypeExtractor.getErrorType(e) == ErrorTypeExtractor.ErrorType.ALREADY_EXISTS) {\n          throw (FileAlreadyExistsException)\n              new FileAlreadyExistsException(\n                  String.format(\"Object '%s' already exists.\", resourceId)\n              ).initCause(e);\n        }\n        throw new IOException(e);\n      }\n    }\n  }\n\n  GoogleCloudStorageItemInfo composeObjects(\n      List<StorageResourceId> sources, StorageResourceId destination, CreateObjectOptions options)\n      throws IOException {\n    LOG.trace(\"composeObjects({}, {}, {})\", sources, destination, options);\n    for (StorageResourceId inputId : sources) {\n      if (!destination.getBucketName().equals(inputId.getBucketName())) {\n        throw new IOException(\n            String.format(\n                \"Bucket doesn't match for source '%s' and destination '%s'!\",\n                inputId, destination));\n      }\n    }\n    Storage.ComposeRequest request =\n        Storage.ComposeRequest.newBuilder()\n            .addSource(\n                sources.stream().map(StorageResourceId::getObjectName).collect(Collectors.toList()))\n            .setTarget(\n                BlobInfo.newBuilder(destination.getBucketName(), destination.getObjectName())\n                    .setContentType(options.getContentType())\n                    .setContentEncoding(options.getContentEncoding())\n                    .setMetadata(encodeMetadata(options.getMetadata()))\n                    .build())\n            .setTargetOptions(\n                Storage.BlobTargetOption.generationMatch(\n                    destination.hasGenerationId()","sourceCodeStart":355,"sourceCodeEnd":391,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java#L355-L391","documentation":"composeObjects concatenates source objects into a destination object using the GCS compose API, which only works among objects in the same bucket. Before building the ComposeRequest the code validates that every source's bucket equals the destination's bucket and throws IOException(\"Bucket doesn't match for source '%s' and destination '%s'!\") otherwise. This is a client-side precondition failure raised before any service call.","triggerScenarios":"Calling compose (directly or via the concat/append code paths that merge small files) where any source StorageResourceId has a different bucket name than the destination — e.g. concatenating gs://bucketA/part-* into gs://bucketB/merged.","commonSituations":"Small-files merge jobs configured with input and output on different buckets; hardcoding a destination bucket that differs from the input bucket; unit tests feeding mixed-bucket resource ids into compose.","solutions":["Keep compose sources and destination in the same bucket","If inputs span buckets, first copy them into the destination bucket, then compose","Fix the job configuration so input/output paths share one bucket"],"exampleFix":"// before\ncomposeObjects(\n  Arrays.asList(new StorageResourceId(\"bucketA\", \"p1\"), new StorageResourceId(\"bucketA\", \"p2\")),\n  new StorageResourceId(\"bucketB\", \"merged\"));\n  // -> Bucket doesn't match for source 'bucketA/p1' and destination 'bucketB/merged'!\n\n// after\nList<StorageResourceId> srcs = Arrays.asList(\n  new StorageResourceId(\"bucketB\", \"p1\"), new StorageResourceId(\"bucketB\", \"p2\"));\ncomposeObjects(srcs, new StorageResourceId(\"bucketB\", \"merged\"));","handlingStrategy":"validation","validationCode":"// Reject mixed-bucket compose inputs before calling the API\nString dstBucket = destination.getBucketName();\nfor (StorageResourceId src : sources) {\n  if (!dstBucket.equals(src.getBucketName())) {\n    throw new IllegalArgumentException(\n        \"compose requires same-bucket sources: \" + src);\n  }\n}\ngcs.composeObjects(sources, destination, options);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep concat/merge inputs and output on one bucket","Copy foreign-bucket inputs over before composing","Assert bucket equality in unit tests of compose call sites"],"tags":["gcs","hadoop-gcp","compose","cross-bucket","validation"],"backgroundTag":"cross-bucket-operation-rejected","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}