{"record":{"id":"5ca8f8ba8604fef9","repo":"apache/hadoop","slug":"path-already-exists-5ca8f8","errorCode":null,"errorMessage":"<path> already exists","messagePattern":"<path> already exists","errorType":"exception","errorClass":"FileAlreadyExistsException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":2172,"sourceCode":"    if (skipList) {\n      probes.remove(StatusProbeEnum.List);\n    }\n\n    // if probes are required -request them and evaluate the result.\n    if (!probes.isEmpty()) {\n      try {\n\n        // get the status or throw an FNFE.\n        FileStatus status = innerGetFileStatus(path, false, probes);\n\n        // if the thread reaches here, there is something at the path\n        if (status.isDirectory()) {\n          // path references a directory: automatic error\n          throw new FileAlreadyExistsException(path + \" is a directory\");\n        }\n        if (!overwrite) {\n          // path references a file and overwrite is disabled\n          throw new FileAlreadyExistsException(path + \" already exists\");\n        }\n        LOG.debug(\"Overwriting file {}\", path);\n      } catch (FileNotFoundException e) {\n        // this means there is nothing at the path; all good.\n      }\n    } else {\n      LOG.debug(\"Skipping all probes with flags:\"\n              + \" createPerf={}, magic={}, ccAvailable={}, cCreate={}, cEtag={}\",\n          createPerf, magic, ccAvailable, cCreate, cEtag);\n    }\n    instrumentation.fileCreated();\n    final BlockOutputStreamStatistics outputStreamStatistics\n        = statisticsContext.newOutputStreamStatistics();\n    PutTracker putTracker =\n        committerIntegration.createTracker(path, key, outputStreamStatistics);\n    String destKey = putTracker.getDestKey();\n\n    EnumSet<WriteObjectFlags> putFlags = options.writeObjectFlags();","sourceCodeStart":2154,"sourceCodeEnd":2190,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2154-L2190","documentation":"The pre-create probes found an existing object at the path and it is a file, but CreateFlag.OVERWRITE is not set, so create raises FileAlreadyExistsException '<path> already exists' instead of clobbering data. With overwrite=true the same probe result is only logged ('Overwriting file') and the write proceeds.","triggerScenarios":"fs.create(path) or fs.create(path, false) - the default no-overwrite form - when an object already exists at path; two writers racing to create the same output file.","commonSituations":"Job reruns without cleanup of previous output; MapReduce/Spark task retries writing the same part file; code ported from systems where create overwrites by default.","solutions":["Pass overwrite=true explicitly when replacing is intended: fs.create(path, true), or delete the object first","For race-safe no-clobber creates use conditional writes (fs.s3a.create.conditional.enabled) or unique per-attempt names","Clean stale output in job setup (e.g. delete the output directory before submission)"],"exampleFix":"// before: defaults to overwrite=false, throws on reruns\nFSDataOutputStream out = fs.create(outputFile);\n\n// after: intent is explicit\nFSDataOutputStream out = fs.create(outputFile, true);","handlingStrategy":"try-catch","validationCode":"// Race-aware no-clobber create\nif (fs.exists(path)) {\n  throw new IllegalStateException(\"output already published: \" + path);\n}\nout = fs.create(path, false);","typeGuard":null,"tryCatchPattern":"Catch FileAlreadyExistsException from create(); for single-writer flows decide overwrite upfront (create(path, true)); for concurrent writers treat it as 'someone else won' and skip or pick a unique name - do not blindly retry.","preventionTips":["Make overwrite intent explicit at every create call site","Clean stale output in job setup","Use unique per-attempt names and publish atomically at the end"],"tags":["s3a","create","file-already-exists","overwrite"],"backgroundTag":"file-already-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}