apache/hadoop · error · AuditOperationRejectedException

Multipart IO request {sdkRequest} rejected {header}

Error message

Multipart IO request {sdkRequest} rejected {header}

What it means

AuditOperationRejectedException thrown by LoggingAuditor's HTTP request hook when multipart uploads are disabled (fs.s3a.multipart.uploads.enabled=false, read at auditor init) and the outgoing SDK request is multipart IO. The auditor acts as a tripwire: writes that need multipart fail fast at request time instead of silently depending on uploads that were administratively disabled.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:445

      if (headerEnabled) {
        // add the referrer header
        httpRequest = httpRequest.toBuilder()
            .appendHeader(HEADER_REFERRER, header)
            .build();
      }

      if (LOG.isDebugEnabled()) {
        LOG.debug("[{}] {} Executing {} with {}; {}",
            currentThreadID(),
            spanId,
            operationName,
            analyzer.analyze(context.request()),
            header);
      }

      // now see if the request is actually a blocked multipart request
      if (!isMultipartUploadEnabled && isRequestMultipartIO(sdkRequest)) {
        throw new AuditOperationRejectedException("Multipart IO request "
            + sdkRequest + " rejected " + header);
      }

      return httpRequest;
    }

    /**
     * For delete requests, attach delete key size as a referrer attribute.
     *
     * @param request the request object.
     */
    private void attachDeleteKeySizeAttribute(SdkRequest request) {

      if (request instanceof DeleteObjectsRequest) {
        int keySize = ((DeleteObjectsRequest) request).delete().objects().size();
        referrer.set(DELETE_KEYS_SIZE, String.valueOf(keySize));
      } else if (request instanceof DeleteObjectRequest) {
        String key = ((DeleteObjectRequest) request).key();

View on GitHub (pinned to 2add963021)

Solutions

1. If multipart writes are intended, set fs.s3a.multipart.uploads.enabled=true (the default)
2. If multipart must stay disabled, stop using the S3A committers and the block output stream on that cluster - they cannot function without it
3. Verify the effective runtime configuration on the nodes doing the writes (bucket-scoped and job-level overrides may differ from the global one)
4. Communicate the policy: with the flag off, S3A on that filesystem is effectively read-mostly for large writes

Example fix

<!-- before: multipart administratively disabled -->
<property><name>fs.s3a.multipart.uploads.enabled</name><value>false</value></property>

<!-- after: multipart required for block output stream + S3A committers -->
<property><name>fs.s3a.multipart.uploads.enabled</name><value>true</value></property>

if (!conf.getBoolean("fs.s3a.multipart.uploads.enabled", true)
    && jobUsesS3ACommitters()) {
  throw new IOException("S3A committers and the block output stream require"
      + " fs.s3a.multipart.uploads.enabled=true");
}

catch AuditOperationRejectedException on write/commit - it is an administrative block; resolve via configuration (enable multipart or stop using multipart-dependent writers), never by retrying

When it happens

Trigger: fs.s3a.multipart.uploads.enabled=false combined with any multipart-issuing operation: writes through the block/fast-upload output stream, or S3A committers (Magic/Directory), which are hard prerequisites for multipart - AbstractS3ACommitter refuses to start when the flag is off.

Common situations: Administrators disabling multipart uploads to block pending-upload debris or committers, after which Hive/Spark jobs with fast upload or the S3A commit protocol still attempt multipart requests; per-bucket or job-level configs differing from the cluster default.

Related errors

• Unable to create OutputStream with the given multipart uploa
• Failed to instantiate class {auditClassname} defined in {key
• Multipart uploads are disabled for the FileSystem, the commi
• Cannot find password option {key}
• SSE-C is enabled but no encryption key was declared in fs.s3

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9b7a4ebb1e24dd2c. Report an issue: GitHub.