{"record":{"id":"25f767ae68bf3af5","repo":"apache/hadoop","slug":"already-exists-25f767","errorCode":null,"errorMessage":"already exists: {}","messagePattern":"already exists: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"warning","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java","lineNumber":3012,"sourceCode":"    /**\n     * Set the progressable object in order to report progress.\n     * @param progressable input Progressable.\n     */\n    public void setProgressable(Progressable progressable) {\n      this.progressable = progressable;\n    }\n    \n    /** \n     * Perform a file sort from a set of input files into an output file.\n     * @param inFiles the files to be sorted\n     * @param outFile the sorted output file\n     * @param deleteInput should the input files be deleted as they are read?\n     * @throws IOException raised on errors performing I/O.\n     */\n    public void sort(Path[] inFiles, Path outFile,\n                     boolean deleteInput) throws IOException {\n      if (fs.exists(outFile)) {\n        throw new IOException(\"already exists: \" + outFile);\n      }\n\n      this.inFiles = inFiles;\n      this.outFile = outFile;\n\n      int segments = sortPass(deleteInput);\n      if (segments > 1) {\n        mergePass(outFile.getParent());\n      }\n    }\n\n    /** \n     * Perform a file sort from a set of input files and return an iterator.\n     * @param inFiles the files to be sorted\n     * @param tempDir the directory where temp files are created during sort\n     * @param deleteInput should the input files be deleted as they are read?\n     * @return iterator the RawKeyValueIterator\n     * @throws IOException raised on errors performing I/O.","sourceCodeStart":2994,"sourceCodeEnd":3030,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java#L2994-L3030","documentation":"SequenceFile.Sorter.sort(Path[], Path, boolean) refuses to overwrite: before any sorting it checks fs.exists(outFile) and throws IOException(\"already exists: <outFile>\") if the destination is present. This is a safety guard against silently clobbering previous sort output, not a corruption signal.","triggerScenarios":"Calling sort() twice with the same output path in one run; rerunning an application that reuses a fixed output location; a previous failed run that wrote the output but never completed cleanup.","commonSituations":"Retried jobs or scripts with deterministic output names; shared working directories on HDFS or local FS; leftovers from crashed runs that were never cleaned.","solutions":["Delete or move the existing output before sorting: if (fs.exists(outFile)) fs.delete(outFile, true).","Write each run to a unique output path (append jobId/timestamp) and rename atomically on success.","Add cleanup of the output location in a finally block when runs fail."],"exampleFix":"// before\nsorter.sort(inFiles, outFile, false);    // throws if outFile exists\n\n// after\nif (fs.exists(outFile) && !fs.delete(outFile, true)) {\n  throw new IOException(\"Cannot replace \" + outFile);\n}\nsorter.sort(inFiles, outFile, false);","handlingStrategy":"validation","validationCode":"if (fs.exists(outFile)) {\n  if (!fs.delete(outFile, true)) {\n    throw new IOException(\"Cannot replace existing output \" + outFile);\n  }\n}\nsorter.sort(inFiles, outFile, deleteInput);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check fs.exists() on every sort/merge output path before starting","Use per-run output names (jobId/timestamp) and publish via atomic rename","Clean working directories in finally blocks when runs fail"],"tags":["sequencefile","sorter","file-exists","filesystem","hadoop"],"backgroundTag":"output-file-already-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}