apache/seatunnel · error · ClickhouseConnectorException

Flush data into clickhouse file error

Error message

Flush data into clickhouse file error

What it means

prepareCommit flushes buffered rows by generating ClickHouse local files (via clickhouse-local) and moving them to the server. Any exception during generateClickhouseLocalFiles or moveClickhouseLocalFileToServer is wrapped in this generic FLUSH_DATA_FAILED exception, with the original exception chained as the cause.

Source

Thrown at seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ClickhouseFileSinkWriter.java:192

        }
    }

    @Override
    public Optional<CKFileCommitInfo> prepareCommit() throws IOException {
        for (FileChannel channel : rowCache.values()) {
            channel.close();
        }
        Map<Shard, List<String>> detachedFiles = new HashMap<>();
        shardTempFile.forEach(
                (shard, path) -> {
                    List<String> clickhouseLocalFiles = null;
                    try {
                        clickhouseLocalFiles = generateClickhouseLocalFiles(path);
                        // move file to server
                        moveClickhouseLocalFileToServer(shard, clickhouseLocalFiles);
                        detachedFiles.put(shard, clickhouseLocalFiles);
                    } catch (Exception e) {
                        throw new ClickhouseConnectorException(
                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,
                                "Flush data into clickhouse file error",
                                e);
                    } finally {
                        if (clickhouseLocalFiles != null && !clickhouseLocalFiles.isEmpty()) {
                            // clear local file
                            clearLocalFileDirectory(clickhouseLocalFiles);
                        }
                    }
                });
        rowCache.clear();
        shardTempFile.clear();
        return Optional.of(new CKFileCommitInfo(detachedFiles));
    }

    @Override
    public void abortPrepare() {}

View on GitHub (pinned to cf67b549a7)

Solutions

1. Inspect the `cause` chained exception — it names the real failure (SSH auth, file-not-exists, process error)
2. Verify SSH connectivity/credentials to the shard node (this flow uses SCP/RSYNC file transfer)
3. Confirm clickhouse-local is installed and runs successfully on the machine producing the files
4. Check disk space and permissions in the local file directory used for temporary files

try {
    // job run triggering prepareCommit flush
} catch (ClickhouseConnectorException e) {
    if ("Flush data into clickhouse file error".equals(e.getMessage()) && e.getCause() != null) {
        log.error("Root cause: ", e.getCause());
    }
}

When it happens

Trigger: prepareCommit invoking generateClickhouseLocalFiles(path) which fails (local file missing, clickhouse-local process error), or moveClickhouseLocalFileToServer(shard, files) failing due to SSH/permission problems on the remote node.

Common situations: SSH credentials or connectivity to the ClickHouse node broken; clickhouse-local not installed or failing on the target machine; disk full on the worker; the generated local file directory missing at flush time.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• COMMON_FLUSH_DATA_FAILED
• clickhouse local file not exists
• FLUSH_DATA_FAILED
• FLUSH_DATA_FAILED
• FLUSH_DATA_FAILED

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/e63ef9353a577bbc. Report an issue: GitHub.