apache/flink · error · IOException

Committing file failed, could not rename

Error message

Committing file failed, could not rename {} -> {}

What it means

Thrown by LocalRecoverableFsDataOutputStream.commit() when the atomic move (Files.move with ATOMIC_MOVE) is not supported and the fallback java.io.File.renameTo also fails. The rename from temp to target could not be completed.

Solutions

  1. Ensure temp file and target file reside on the same filesystem/volume.
  2. Fix permissions on the target directory for the Flink user.
  3. Close any handles on the target file (especially on Windows) before commit.
  4. If cross-volume is unavoidable, reconfigure temp dir to share the target's volume.
Defensive patterns

Strategy: try-catch

Validate before calling

void ensureSameVolume(File src, File dest) throws IOException {
    if (!src.toPath().getParent().equals(dest.toPath().getParent())
        && !src.getAbsolutePath().startsWith(rootsOf(dest)))
        throw new IOException("temp and target likely on different volumes");
}

Try / catch

try {
    committer.commit();
} catch (IOException e) {
    if (e.getMessage().startsWith("Committing file failed, could not rename")) {
        // move temp to same volume as target, or fix perms, then retry
    }
    throw e;
}

Prevention

When it happens

Trigger: Commit() on a filesystem/OS where ATOMIC_MOVE is unsupported (catches UnsupportedOperationException/AtomicMoveNotSupportedException), and the fallback renameTo(dest) returns false.

Common situations: Cross-device rename (temp and target on different mount points); permissions on the target directory; target path locked/open on Windows; filesystem that disallows rename.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/a59fb45c700b1a67. Report an issue: GitHub.

Appendix: source

Thrown at flink-core/src/main/java/org/apache/flink/core/fs/local/LocalRecoverableFsDataOutputStream.java:165

        @Override
        public void commit() throws IOException {
            final File src = recoverable.tempFile();
            final File dest = recoverable.targetFile();

            // sanity check
            if (src.length() != recoverable.offset()) {
                // something was done to this file since the committer was created.
                // this is not the "clean" case
                throw new IOException("Cannot clean commit: File has trailing junk data.");
            }

            // rather than fall into default recovery, handle errors explicitly
            // in order to improve error messages
            try {
                Files.move(src.toPath(), dest.toPath(), StandardCopyOption.ATOMIC_MOVE);
            } catch (UnsupportedOperationException | AtomicMoveNotSupportedException e) {
                if (!src.renameTo(dest)) {
                    throw new IOException(
                            "Committing file failed, could not rename " + src + " -> " + dest);
                }
            } catch (FileAlreadyExistsException e) {
                throw new IOException(
                        "Committing file failed. Target file already exists: " + dest);
            }
        }

        @Override
        public void commitAfterRecovery() throws IOException {
            final File src = recoverable.tempFile();
            final File dest = recoverable.targetFile();
            final long expectedLength = recoverable.offset();

            if (src.exists()) {
                if (src.length() > expectedLength) {
                    // can happen if we co from persist to recovering for commit directly
                    // truncate the trailing junk away

View on GitHub (pinned to 2f3c205e92)