{"record":{"id":"9b7a4ebb1e24dd2c","repo":"apache/hadoop","slug":"multipart-io-request-sdkrequest-rejected-header","errorCode":null,"errorMessage":"Multipart IO request {sdkRequest} rejected {header}","messagePattern":"Multipart IO request (.+?) rejected (.+?)","errorType":"exception","errorClass":"AuditOperationRejectedException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java","lineNumber":445,"sourceCode":"      if (headerEnabled) {\n        // add the referrer header\n        httpRequest = httpRequest.toBuilder()\n            .appendHeader(HEADER_REFERRER, header)\n            .build();\n      }\n\n      if (LOG.isDebugEnabled()) {\n        LOG.debug(\"[{}] {} Executing {} with {}; {}\",\n            currentThreadID(),\n            spanId,\n            operationName,\n            analyzer.analyze(context.request()),\n            header);\n      }\n\n      // now see if the request is actually a blocked multipart request\n      if (!isMultipartUploadEnabled && isRequestMultipartIO(sdkRequest)) {\n        throw new AuditOperationRejectedException(\"Multipart IO request \"\n            + sdkRequest + \" rejected \" + header);\n      }\n\n      return httpRequest;\n    }\n\n    /**\n     * For delete requests, attach delete key size as a referrer attribute.\n     *\n     * @param request the request object.\n     */\n    private void attachDeleteKeySizeAttribute(SdkRequest request) {\n\n      if (request instanceof DeleteObjectsRequest) {\n        int keySize = ((DeleteObjectsRequest) request).delete().objects().size();\n        referrer.set(DELETE_KEYS_SIZE, String.valueOf(keySize));\n      } else if (request instanceof DeleteObjectRequest) {\n        String key = ((DeleteObjectRequest) request).key();","sourceCodeStart":427,"sourceCodeEnd":463,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java#L427-L463","documentation":"AuditOperationRejectedException thrown by LoggingAuditor's HTTP request hook when multipart uploads are disabled (fs.s3a.multipart.uploads.enabled=false, read at auditor init) and the outgoing SDK request is multipart IO. The auditor acts as a tripwire: writes that need multipart fail fast at request time instead of silently depending on uploads that were administratively disabled.","triggerScenarios":"fs.s3a.multipart.uploads.enabled=false combined with any multipart-issuing operation: writes through the block/fast-upload output stream, or S3A committers (Magic/Directory), which are hard prerequisites for multipart - AbstractS3ACommitter refuses to start when the flag is off.","commonSituations":"Administrators disabling multipart uploads to block pending-upload debris or committers, after which Hive/Spark jobs with fast upload or the S3A commit protocol still attempt multipart requests; per-bucket or job-level configs differing from the cluster default.","solutions":["If multipart writes are intended, set fs.s3a.multipart.uploads.enabled=true (the default)","If multipart must stay disabled, stop using the S3A committers and the block output stream on that cluster - they cannot function without it","Verify the effective runtime configuration on the nodes doing the writes (bucket-scoped and job-level overrides may differ from the global one)","Communicate the policy: with the flag off, S3A on that filesystem is effectively read-mostly for large writes"],"exampleFix":"<!-- before: multipart administratively disabled -->\n<property><name>fs.s3a.multipart.uploads.enabled</name><value>false</value></property>\n\n<!-- after: multipart required for block output stream + S3A committers -->\n<property><name>fs.s3a.multipart.uploads.enabled</name><value>true</value></property>","handlingStrategy":"validation","validationCode":"if (!conf.getBoolean(\"fs.s3a.multipart.uploads.enabled\", true)\n    && jobUsesS3ACommitters()) {\n  throw new IOException(\"S3A committers and the block output stream require\"\n      + \" fs.s3a.multipart.uploads.enabled=true\");\n}","typeGuard":null,"tryCatchPattern":"catch AuditOperationRejectedException on write/commit - it is an administrative block; resolve via configuration (enable multipart or stop using multipart-dependent writers), never by retrying","preventionTips":["Treat fs.s3a.multipart.uploads.enabled=false as a write-restriction policy and communicate it to job owners","Gate committer-based jobs on this flag in job setup","Monitor logs for AuditOperationRejectedException to catch misrouted jobs early"],"tags":["s3a","hadoop-aws","multipart-upload","auditing","configuration","committer"],"backgroundTag":"multipart-upload-disabled","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}