apache/seatunnel · warning

rename file failed, will continue move file, but maybe…

Error message

rename file failed, will continue move file, but maybe cause file conflict

What it means

When preparing ClickHouse local part files for transfer, ClickhouseFileSinkWriter renames each file to include the subtask index to avoid name conflicts between subtasks. If File.renameTo fails (typically a cross-filesystem move or permission issue), it logs this warning and returns the original file name, which may collide with files from other subtasks.

Solutions

1. Configure the ClickHouse local file paths so temp and target directories are on the same filesystem/device.
2. Ensure the target directory is writable by the SeaTunnel process user and the target filename does not already exist.
3. Clean up stale part files from previous failed runs to avoid name conflicts.

Example fix

// before
if (f.renameTo(newFile)) {
    return newFile;
} else {
    log.warn("rename file failed, ...");
    return f;
}
// after (Java 7+ atomic move with fallback)
try {
    Files.move(f.toPath(), newFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
    return newFile;
} catch (IOException e) {
    log.warn("rename file failed: {}", e.getMessage());
    return f;
}

// ensure same filesystem before run
File src = ..., dst = ...;
if (!src.toPath().getFileSystem(env).equals(dst.toPath().getFileSystem(env))) {
    throw new IllegalStateException("source and target must share a filesystem");
}

When it happens

Trigger: generateClickhouseLocalFiles calls f.renameTo(newFile) and the OS rename fails — e.g. source and target on different mount points, target file exists, or insufficient write permission on the target directory.

Common situations: SeaTunnel tmp directory and ClickHouse local data directory on different filesystems; multiple subtasks sharing a working dir where the target name already exists; container volume mounts with restricted permissions.

Related errors

• File already exists
• Agent config is not a readable file
• array inject error, unsupported data type: " + type
• Both 'mappings' and 'schema_config' are present…
• Cannot find password of shard " +…

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/6b32645622c2a6ac. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ClickhouseFileSinkWriter.java:379

                    ClickhouseConnectorErrorCode.FILE_NOT_EXISTS,
                    "clickhouse local file not exists");
        }
        return Arrays.stream(files)
                .filter(File::isDirectory)
                .filter(f -> !"detached".equals(f.getName()))
                .map(
                        f -> {
                            File newFile =
                                    new File(
                                            f.getParent()
                                                    + "/"
                                                    + f.getName()
                                                    + "_"
                                                    + context.getIndexOfSubtask());
                            if (f.renameTo(newFile)) {
                                return newFile;
                            } else {
                                log.warn(
                                        "rename file failed, will continue move file, but maybe cause file conflict");
                                return f;
                            }
                        })
                .map(File::getAbsolutePath)
                .collect(Collectors.toList());
    }

    private void moveClickhouseLocalFileToServer(Shard shard, List<String> clickhouseLocalFiles) {
        String hostAddress = shard.getNode().getHost();
        String user = readerOption.getNodeUser().getOrDefault(hostAddress, "root");
        String password = readerOption.getNodePassword().getOrDefault(hostAddress, null);
        String keyPath = readerOption.getKeyPath();
        FileTransfer fileTransfer =
                FileTransferFactory.createFileTransfer(
                        this.readerOption.getCopyMethod(), hostAddress, user, password, keyPath);
        fileTransfer.init();
        int randomPath = threadLocalRandom.nextInt(shardLocalDataPaths.get(shard).size());

View on GitHub (pinned to cf67b549a7)