{"record":{"id":"e63ef9353a577bbc","repo":"apache/seatunnel","slug":"flush-data-into-clickhouse-file-error","errorCode":null,"errorMessage":"Flush data into clickhouse file error","messagePattern":"Flush data into clickhouse file error","errorType":"exception","errorClass":"ClickhouseConnectorException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ClickhouseFileSinkWriter.java","lineNumber":192,"sourceCode":"        }\n    }\n\n    @Override\n    public Optional<CKFileCommitInfo> prepareCommit() throws IOException {\n        for (FileChannel channel : rowCache.values()) {\n            channel.close();\n        }\n        Map<Shard, List<String>> detachedFiles = new HashMap<>();\n        shardTempFile.forEach(\n                (shard, path) -> {\n                    List<String> clickhouseLocalFiles = null;\n                    try {\n                        clickhouseLocalFiles = generateClickhouseLocalFiles(path);\n                        // move file to server\n                        moveClickhouseLocalFileToServer(shard, clickhouseLocalFiles);\n                        detachedFiles.put(shard, clickhouseLocalFiles);\n                    } catch (Exception e) {\n                        throw new ClickhouseConnectorException(\n                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n                                \"Flush data into clickhouse file error\",\n                                e);\n                    } finally {\n                        if (clickhouseLocalFiles != null && !clickhouseLocalFiles.isEmpty()) {\n                            // clear local file\n                            clearLocalFileDirectory(clickhouseLocalFiles);\n                        }\n                    }\n                });\n        rowCache.clear();\n        shardTempFile.clear();\n        return Optional.of(new CKFileCommitInfo(detachedFiles));\n    }\n\n    @Override\n    public void abortPrepare() {}\n","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ClickhouseFileSinkWriter.java#L174-L210","documentation":"prepareCommit flushes buffered rows by generating ClickHouse local files (via clickhouse-local) and moving them to the server. Any exception during generateClickhouseLocalFiles or moveClickhouseLocalFileToServer is wrapped in this generic FLUSH_DATA_FAILED exception, with the original exception chained as the cause.","triggerScenarios":"prepareCommit invoking generateClickhouseLocalFiles(path) which fails (local file missing, clickhouse-local process error), or moveClickhouseLocalFileToServer(shard, files) failing due to SSH/permission problems on the remote node.","commonSituations":"SSH credentials or connectivity to the ClickHouse node broken; clickhouse-local not installed or failing on the target machine; disk full on the worker; the generated local file directory missing at flush time.","solutions":["Inspect the `cause` chained exception — it names the real failure (SSH auth, file-not-exists, process error)","Verify SSH connectivity/credentials to the shard node (this flow uses SCP/RSYNC file transfer)","Confirm clickhouse-local is installed and runs successfully on the machine producing the files","Check disk space and permissions in the local file directory used for temporary files"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    // job run triggering prepareCommit flush\n} catch (ClickhouseConnectorException e) {\n    if (\"Flush data into clickhouse file error\".equals(e.getMessage()) && e.getCause() != null) {\n        log.error(\"Root cause: \", e.getCause());\n    }\n}","preventionTips":["Verify SSH access from worker to each shard node before running the job","Ensure clickhouse-local is installed and functional on generating machines","Monitor disk space in localFileDir"],"tags":["clickhouse","flush","io"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}