{"record":{"id":"a1c41f8e6f266107","repo":"apache/hadoop","slug":"multi-part-uploader-not-supported-for-client-side","errorCode":null,"errorMessage":"Multi-part uploader not supported for Client side encryption.","messagePattern":"Multi-part uploader not supported for Client side encryption\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":5590,"sourceCode":"    OpenFileSupport.OpenFileInformation fileInformation =\n        openFileHelper.prepareToOpenFile(\n            path,\n            parameters,\n            getDefaultBlockSize());\n    CompletableFuture<FSDataInputStream> result = new CompletableFuture<>();\n    unboundedThreadPool.submit(() ->\n        LambdaUtils.eval(result,\n            () -> executeOpen(path, fileInformation)));\n    return result;\n  }\n\n  @Override\n  @AuditEntryPoint\n  public S3AMultipartUploaderBuilder createMultipartUploader(\n      final Path basePath)\n      throws IOException {\n    if(isCSEEnabled) {\n      throw new UnsupportedOperationException(\"Multi-part uploader not \"\n          + \"supported for Client side encryption.\");\n    }\n    final Path path = makeQualified(basePath);\n    try (AuditSpan span = entryPoint(MULTIPART_UPLOAD_INSTANTIATED, path)) {\n      StoreContext ctx = createStoreContext();\n      return new S3AMultipartUploaderBuilder(this,\n          createWriteOperationHelper(span),\n          ctx,\n          path,\n          statisticsContext.createMultipartUploaderStatistics());\n    }\n  }\n\n  /**\n   * Build an immutable store context.\n   * If called while the FS is being initialized,\n   * some of the context will be incomplete.\n   * new store context instances should be created as appropriate.","sourceCodeStart":5572,"sourceCodeEnd":5608,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L5572-L5608","documentation":"createMultipartUploader() checks isCSEEnabled - set in initialize() when the configured encryption algorithm resolves to a client-side-encryption method (CSE-KMS/CSE-CUSTOM via fs.s3a.encryption.algorithm or the deprecated fs.s3a.server-side-encryption-algorithm) - and throws UnsupportedOperationException, because the MultipartUploader API cannot produce CSE-compatible uploads. Non-CSE filesystems get a working S3AMultipartUploaderBuilder; only CSE configurations are rejected.","triggerScenarios":"Calling fs.createMultipartUploader(path), or running tools built on the MultipartUploader API (e.g. distcp multipart copy paths), on a filesystem whose encryption algorithm config selects CSE.","commonSituations":"CSE enabled for compliance on a bucket, then a tool or workflow that uses multipart upload APIs runs against the same configuration; core-site.xml with CSE settings shared across heterogeneous workloads.","solutions":["Unset the CSE algorithm for workloads that need the MultipartUploader API (or move to SSE if policy allows)","Avoid the MultipartUploader API on CSE buckets - plain FSDataOutputStream writes through S3ABlockOutputStream still work","Separate buckets/configurations for CSE-only and multipart-uploader workloads"],"exampleFix":"<!-- before: CSE enabled, blocks multipart uploader -->\n<property><name>fs.s3a.encryption.algorithm</name><value>CSE-KMS</value></property>\n<property><name>fs.s3a.client.side.encryption.key</name><value>...</value></property>\n\n<!-- after: SSE-KMS keeps server-side encryption compatible with multipart upload -->\n<property><name>fs.s3a.encryption.algorithm</name><value>SSE-KMS</value></property>","handlingStrategy":"validation","validationCode":"static boolean cseEnabled(Configuration conf) {\n  String alg = conf.get(\"fs.s3a.encryption.algorithm\",\n      conf.get(\"fs.s3a.server-side-encryption-algorithm\", \"\"));\n  return alg.startsWith(\"CSE\");\n}\n\nif (cseEnabled(conf)) {\n  // skip the MultipartUploader API; use plain stream writes\n}","typeGuard":null,"tryCatchPattern":"Catch UnsupportedOperationException from createMultipartUploader() and fall back to normal FSDataOutputStream-based writes; surface the CSE/multipart incompatibility in configuration rather than disabling encryption silently.","preventionTips":["Keep CSE and multipart-uploader workloads on separate buckets/configurations","Detect CSE configuration before invoking MultipartUploader-based tooling","When encryption-at-rest and multipart APIs are both required, prefer SSE algorithms over CSE"],"tags":["s3a","client-side-encryption","multipart-upload","unsupported-operation"],"backgroundTag":"feature-incompatible-configuration","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}