{"record":{"id":"3635a79f92429bf1","repo":"apache/hadoop","slug":"pathconflict-3635a7","errorCode":"PathConflict","errorMessage":"The specified path, or an element of the path, exists and its resource type is invalid for this operation.","messagePattern":"The specified path, or an element of the path, exists and its resource type is invalid for this operation\\.","errorType":"http","errorClass":"AbfsRestOperationException","httpStatus":409,"severity":"error","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java","lineNumber":518,"sourceCode":"      final ContextEncryptionAdapter contextEncryptionAdapter,\n      final TracingContext tracingContext) throws AzureBlobFileSystemException {\n    AbfsRestOperation op;\n    if (isFileCreation) {\n      if (getAbfsConfiguration().getIsCreateIdempotencyEnabled()) {\n        AbfsRestOperation statusOp = null;\n        try {\n          // Check if the file already exists by calling GetPathStatus\n          statusOp = getPathStatus(path, tracingContext, null, false);\n        } catch (AbfsRestOperationException ex) {\n          // If the path does not exist, continue with file creation\n          // For other errors, rethrow the exception\n          if (ex.getStatusCode() != HTTP_NOT_FOUND) {\n            throw ex;\n          }\n        }\n        // If the file exists and overwrite is not allowed, throw conflict\n        if (statusOp != null && statusOp.hasResult() && !overwrite) {\n          throw new AbfsRestOperationException(\n              HTTP_CONFLICT,\n              AzureServiceErrorCode.PATH_CONFLICT.getErrorCode(),\n              PATH_EXISTS,\n              null);\n        } else {\n          // Proceed with file creation (force overwrite = true)\n          op = createFile(path, true, permissions, isAppendBlob, eTag,\n              contextEncryptionAdapter, tracingContext);\n        }\n      } else {\n        op = createFile(path, overwrite, permissions, isAppendBlob, eTag,\n            contextEncryptionAdapter, tracingContext);\n      }\n    } else {\n      // Create a directory with the specified parameters\n      op = createDirectory(path, permissions, isAppendBlob, eTag,\n          contextEncryptionAdapter, tracingContext);\n    }","sourceCodeStart":500,"sourceCodeEnd":536,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java#L500-L536","documentation":"In conditionalCreateOverwriteFile on a non-HNS account, the client first probes the path with GetPathStatus; if something already exists there and overwrite=false, it throws AbfsRestOperationException with HTTP 409, AzureServiceErrorCode.PathConflict, and the service-style message 'The specified path, or an element of the path, exists and its resource type is invalid for this operation.' The client synthesizes the same conflict the DFS endpoint would return, before even attempting the create.","triggerScenarios":"fs.create(path, false) (overwrite=false) when getPathStatus succeeds for that path on a flat-namespace account — any file or blob already occupies the target.","commonSituations":"Job reruns without cleaning prior output; leftover temp files from a failed committer; the overwrite flag not propagated (FSDataOutputStream created through APIs defaulting to overwrite=false).","solutions":["Pass overwrite=true when clobbering prior output is acceptable: fs.create(path, true)","Delete or rename the pre-existing path before creating","Catch the 409/PathConflict and implement idempotent 'already done' logic for restarted jobs"],"exampleFix":"// before: refuses to clobber -> AbfsRestOperationException 409 PathConflict\ntry (FSDataOutputStream out = fs.create(outPath, false)) { ... }\n\n// after: explicit overwrite\ntry (FSDataOutputStream out = fs.create(outPath, true)) { ... }","handlingStrategy":"validation","validationCode":"if (fs.exists(path)) {\n  if (!overwriteAllowed) throw new IllegalStateException(\"output already exists: \" + path);\n  fs.delete(path, false);\n}\ntry (FSDataOutputStream out = fs.create(path, true)) { ... }","typeGuard":null,"tryCatchPattern":"try {\n  fs.create(path, false);\n} catch (AbfsRestOperationException e) {\n  if (e.getStatusCode() == 409\n      && AzureServiceErrorCode.PATH_CONFLICT.getErrorCode().equals(e.getErrorCode())) {\n    // target occupied: decide overwrite vs fail\n  } else { throw e; }\n}","preventionTips":["Decide the overwrite policy explicitly at every create site","Clean job output directories between runs","Treat 409 PathConflict on create as an expected, handleable outcome in job restart logic"],"tags":["azure","abfs","conflict","http-409","create","blob-endpoint"],"backgroundTag":"path-already-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}