apache/hadoop · error · ConcurrentWriteOperationDetectedException

Parallel access to the create path detected. Failing request

Error message

Parallel access to the create path detected. Failing request due to precondition failure

What it means

Second race detector in conditionalCreateOverwriteFile: after reading the eTag, the client re-issues the create conditioned on If-Match <eTag>. HTTP 412 Precondition Failed means the object changed between the status call and the create (a different eTag now), so the write is aborted with ConcurrentWriteOperationDetectedException to preserve single-writer semantics instead of overwriting a stranger's data.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:688

        if (isExplicitDir) {
          throw new AbfsRestOperationException(HTTP_CONFLICT,
              AzureServiceErrorCode.PATH_CONFLICT.getErrorCode(),
              PATH_EXISTS,
              null);
        }

        String eTag = extractEtagHeader(op.getResult());

        try {
          // overwrite only if eTag matches with the file properties fetched before
          op = createPathRestOp(relativePath, true, true,
              isAppendBlob, eTag, contextEncryptionAdapter, tracingContext);
        } catch (AbfsRestOperationException ex) {
          if (ex.getStatusCode() == HTTP_PRECON_FAILED) {
            // Is a parallel access case, as file with eTag was just queried
            // and precondition failure can happen only when another file with
            // different etag got created.
            throw new ConcurrentWriteOperationDetectedException("Parallel access to the create path detected. Failing request "
                + "due to precondition failure");
          } else {
            throw ex;
          }
        }
      } else {
        throw e;
      }
    }
    return op;
  }

  /**
   * Get Rest Operation for API
   * <a href="../../../../site/markdown/blobEndpoint.md#lease-blob">Lease Blob</a>.
   * @param path on which lease has to be acquired.
   * @param duration for which lease has to be acquired.
   * @param tracingContext for tracing the service call.

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the whole create sequence — the next round fetches a fresh eTag and either succeeds or detects a stable state
2. Guarantee a single writer per output path (unique per-attempt names, commit coordinator)
3. Disable speculative duplication for these writes

try {
  fs.create(path, false).close();
} catch (ConcurrentWriteOperationDetectedException e) {
  // eTag raced (412): refetch state and retry the whole create sequence
  sleepBackoff();
  fs.create(path, false).close();
}

When it happens

Trigger: Non-HNS account: between GetPathStatus (eTag fetch) and the conditional create, another writer replaced or re-created the blob; the service returns 412 and the client maps it to this exception.

Common situations: Two tasks or jobs with identical output paths; overwrite races between appenders and creators; commit protocols where task pre-commit and job commit create the same file.

Related errors

• Parallel access to the create path detected. Failing request
• Cannot create file {} because parent folder does not exist.
• PathConflict
• FNS-Blob rename was not successful for source and destinatio
• Flush without blockIds not supported on Blob Endpoint

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/34c0f97dbccaa114. Report an issue: GitHub.