{"record":{"id":"6bc602408c098a6f","repo":"apache/flink","slug":"committing-file-failed-target-file-already-exists","errorCode":null,"errorMessage":"Committing file failed. Target file already exists: {}","messagePattern":"Committing file failed\\. Target file already exists: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/core/fs/local/LocalRecoverableFsDataOutputStream.java","lineNumber":169,"sourceCode":"\n            // sanity check\n            if (src.length() != recoverable.offset()) {\n                // something was done to this file since the committer was created.\n                // this is not the \"clean\" case\n                throw new IOException(\"Cannot clean commit: File has trailing junk data.\");\n            }\n\n            // rather than fall into default recovery, handle errors explicitly\n            // in order to improve error messages\n            try {\n                Files.move(src.toPath(), dest.toPath(), StandardCopyOption.ATOMIC_MOVE);\n            } catch (UnsupportedOperationException | AtomicMoveNotSupportedException e) {\n                if (!src.renameTo(dest)) {\n                    throw new IOException(\n                            \"Committing file failed, could not rename \" + src + \" -> \" + dest);\n                }\n            } catch (FileAlreadyExistsException e) {\n                throw new IOException(\n                        \"Committing file failed. Target file already exists: \" + dest);\n            }\n        }\n\n        @Override\n        public void commitAfterRecovery() throws IOException {\n            final File src = recoverable.tempFile();\n            final File dest = recoverable.targetFile();\n            final long expectedLength = recoverable.offset();\n\n            if (src.exists()) {\n                if (src.length() > expectedLength) {\n                    // can happen if we co from persist to recovering for commit directly\n                    // truncate the trailing junk away\n                    try (FileOutputStream fos = new FileOutputStream(src, true)) {\n                        fos.getChannel().truncate(expectedLength);\n                    }\n                } else if (src.length() < expectedLength) {","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/core/fs/local/LocalRecoverableFsDataOutputStream.java#L151-L187","documentation":"Thrown by LocalRecoverableFsDataOutputStream.commit() when the atomic move raises FileAlreadyExistsException — i.e. the target file already exists at commit time. This is a distinct, more specific failure than a generic rename error.","triggerScenarios":"Calling commit() when the target file already exists on disk, so Files.move(src, dest, ATOMIC_MOVE) refuses to overwrite.","commonSituations":"Duplicate commit; a previous successful commit left the target; concurrent writers committing to the same target; a prior run's output not cleaned.","solutions":["Ensure each target is committed exactly once; deduplicate or guard commit calls.","Delete or overwrite the existing target before commit if re-commit is intended.","In recovery, prefer commitAfterRecovery() which handles existing-target cases."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"void ensureTargetAbsent(LocalRecoverable r) throws IOException {\n    if (r.targetFile().exists())\n        throw new FileAlreadyExistsException(\"target exists; remove or use unique name: \" + r.targetFile());\n}","typeGuard":null,"tryCatchPattern":"try {\n    committer.commit();\n} catch (IOException e) {\n    if (e.getMessage().contains(\"Target file already exists\")) {\n        // deduplicate: treat as already-committed, or pick unique target\n    } else throw e;\n}","preventionTips":["Commit each target exactly once.","Use unique target names per attempt to avoid collisions.","Clean targets before re-running jobs that commit."],"tags":["filesystem","local-fs","recoverable-writer","commit","duplicate"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}