apache/hadoop · error · ConcurrentWriteOperationDetectedException

Parallel access to the create path detected. Failing request

Error message

Parallel access to the create path detected. Failing request as the path which existed before gives not found error

What it means

First race detector inside conditionalCreateOverwriteFile: the initial create with overwrite=false returns 409, so the client re-checks with GetPathStatus to fetch the eTag — but that call returns 404. The object that caused the conflict disappeared between the two calls, so the client throws ConcurrentWriteOperationDetectedException (an AzureBlobFileSystemException) rather than retrying blindly, honoring single-writer semantics.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:661

          contextEncryptionAdapter, tracingContext);
    }
    AbfsRestOperation op;
    try {
      // Trigger a creation with overwrite=false first so that eTag fetch can be
      // avoided for cases when no pre-existing file is present (major portion
      // of create file traffic falls into the case of no pre-existing file).
      op = createPathRestOp(relativePath, true, false,
          isAppendBlob, null, contextEncryptionAdapter, tracingContext);
    } catch (AbfsRestOperationException e) {
      if (e.getStatusCode() == HTTP_CONFLICT) {
        // File pre-exists, fetch eTag
        try {
          op = getPathStatus(relativePath, tracingContext, null, false);
        } catch (AbfsRestOperationException ex) {
          if (ex.getStatusCode() == HTTP_NOT_FOUND) {
            // Is a parallel access case, as file which was found to be
            // present went missing by this request.
            throw new ConcurrentWriteOperationDetectedException("Parallel access to the create path detected. Failing request "
                + "as the path which existed before gives not found error");
          } else {
            throw ex;
          }
        }

        // If present as an explicit empty directory, we should throw conflict exception.
        boolean isExplicitDir = checkIsDir(op.getResult());
        if (isExplicitDir) {
          throw new AbfsRestOperationException(HTTP_CONFLICT,
              AzureServiceErrorCode.PATH_CONFLICT.getErrorCode(),
              PATH_EXISTS,
              null);
        }

        String eTag = extractEtagHeader(op.getResult());

        try {

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the create from the caller — the next attempt starts from a clean state
2. Serialize writers on the path (application-level lock, single committer, unique per-attempt file names)
3. Disable speculative execution for tasks that write deterministic output paths
4. Exclude active output prefixes from storage lifecycle/delete policies

try {
  fs.create(path, false).close();
} catch (ConcurrentWriteOperationDetectedException e) {
  // 409-then-404 race: state changed under us; one retry usually converges
  fs.create(path, false).close();
}

When it happens

Trigger: Two actors racing on the same path on a non-HNS account: writer A's create gets 409 because writer B's file exists; by the time A's status check runs, B (or a deleter) has removed it — 409 followed by 404 in one sequence.

Common situations: Concurrent jobs or speculative tasks writing identical output paths; commit-time races between task attempts and cleanup; external lifecycle-management policies deleting blobs mid-flight.

Related errors

• Cannot create file {} because parent folder does not exist.
• PathConflict
• Parallel access to the create path detected. Failing request
• PathNotFound
• FNS-Blob rename was not successful for source and destinatio

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/14bb60901fbebda2. Report an issue: GitHub.