apache/flink · error · IOException

Committing file failed, could not rename {} -> {}

Error message

Committing file failed, could not rename {} -> {}

What it means

Thrown by LocalRecoverableFsDataOutputStream.commit() when the atomic move (Files.move with ATOMIC_MOVE) is not supported and the fallback java.io.File.renameTo also fails. The rename from temp to target could not be completed.

Source

Thrown at flink-core/src/main/java/org/apache/flink/core/fs/local/LocalRecoverableFsDataOutputStream.java:165

        @Override
        public void commit() throws IOException {
            final File src = recoverable.tempFile();
            final File dest = recoverable.targetFile();

            // sanity check
            if (src.length() != recoverable.offset()) {
                // something was done to this file since the committer was created.
                // this is not the "clean" case
                throw new IOException("Cannot clean commit: File has trailing junk data.");
            }

            // rather than fall into default recovery, handle errors explicitly
            // in order to improve error messages
            try {
                Files.move(src.toPath(), dest.toPath(), StandardCopyOption.ATOMIC_MOVE);
            } catch (UnsupportedOperationException | AtomicMoveNotSupportedException e) {
                if (!src.renameTo(dest)) {
                    throw new IOException(
                            "Committing file failed, could not rename " + src + " -> " + dest);
                }
            } catch (FileAlreadyExistsException e) {
                throw new IOException(
                        "Committing file failed. Target file already exists: " + dest);
            }
        }

        @Override
        public void commitAfterRecovery() throws IOException {
            final File src = recoverable.tempFile();
            final File dest = recoverable.targetFile();
            final long expectedLength = recoverable.offset();

            if (src.exists()) {
                if (src.length() > expectedLength) {
                    // can happen if we co from persist to recovering for commit directly
                    // truncate the trailing junk away

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure temp file and target file reside on the same filesystem/volume.
2. Fix permissions on the target directory for the Flink user.
3. Close any handles on the target file (especially on Windows) before commit.
4. If cross-volume is unavoidable, reconfigure temp dir to share the target's volume.

void ensureSameVolume(File src, File dest) throws IOException {
    if (!src.toPath().getParent().equals(dest.toPath().getParent())
        && !src.getAbsolutePath().startsWith(rootsOf(dest)))
        throw new IOException("temp and target likely on different volumes");
}

try {
    committer.commit();
} catch (IOException e) {
    if (e.getMessage().startsWith("Committing file failed, could not rename")) {
        // move temp to same volume as target, or fix perms, then retry
    }
    throw e;
}

When it happens

Trigger: Commit() on a filesystem/OS where ATOMIC_MOVE is unsupported (catches UnsupportedOperationException/AtomicMoveNotSupportedException), and the fallback renameTo(dest) returns false.

Common situations: Cross-device rename (temp and target on different mount points); permissions on the target directory; target path locked/open on Windows; filesystem that disallows rename.

Related errors

• Cannot clean commit: File has trailing junk data.
• Committing file failed. Target file already exists: {}
• File Not Found: {}
• Missing data in tmp file: {}
• Could not commit file from %s to %s

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/a59fb45c700b1a67. Report an issue: GitHub.