{"record":{"id":"5a1b189b56409f0d","repo":"apache/seatunnel","slug":"flush-data-failed-5a1b18","errorCode":"FLUSH_DATA_FAILED","errorMessage":"Flush data to this file [%s] failed","messagePattern":"Flush data to this file \\[(.+?)\\] failed","errorType":"error_code","errorClass":"FileConnectorException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/CsvWriteStrategy.java","lineNumber":133,"sourceCode":"            if (isFirstWrite.get(filePath)) {\n                isFirstWrite.put(filePath, false);\n            } else {\n                fsDataOutputStream.write(rowDelimiter.getBytes(charset));\n            }\n            fsDataOutputStream.write(serializationSchema.serialize(safeProjectedRow(seaTunnelRow)));\n        } catch (IOException e) {\n            throw CommonError.fileOperationFailed(\"CsvFile\", \"write\", filePath, e);\n        }\n    }\n\n    @Override\n    public void finishAndCloseFile() {\n        beingWrittenOutputStream.forEach(\n                (key, value) -> {\n                    try {\n                        value.flush();\n                    } catch (IOException e) {\n                        throw new FileConnectorException(\n                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n                                String.format(\"Flush data to this file [%s] failed\", key),\n                                e);\n                    } finally {\n                        try {\n                            value.close();\n                        } catch (IOException e) {\n                            log.error(\"error when close output stream {}\", key, e);\n                        }\n                    }\n                    needMoveFiles.put(key, getTargetLocation(key));\n                });\n        beingWrittenOutputStream.clear();\n        isFirstWrite.clear();\n    }\n\n    @Override\n    public FSDataOutputStream getOrCreateOutputStream(@NonNull String filePath) {","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/CsvWriteStrategy.java#L115-L151","documentation":"Thrown by CsvWriteStrategy.finishAndCloseFile when flushing buffered bytes to the underlying Hadoop FSDataOutputStream for a CSV output file fails with an IOException. This happens at file finalization, so the file being written is likely incomplete or corrupt. The original IOException is chained as the cause.","triggerScenarios":"value.flush() throws IOException while iterating beingWrittenOutputStream during finishAndCloseFile — typically disk full, HDFS/DataNode write failure, or the stream/file was already closed or deleted.","commonSituations":"Target filesystem (HDFS/S3/OSS) out of quota or disk full; transient network partition to remote storage during checkpoint/close; permissions revoked mid-write; writing to a file deleted by another job.","solutions":["Check the chained cause (e.getCause()) for the real IOException (disk full, permission, connection) and free space or fix storage access","Verify connectivity and quota on the target filesystem (hdfs dfs -put a test file, check S3/OSS credentials and bucket quota)","Ensure only one job writes to the same output path; enable overwrite or use unique transaction/step directories","Retry the job — flush failures on remote FS are often transient"],"exampleFix":"// before\ntry { value.flush(); } catch (IOException e) { e.printStackTrace(); }\n// after\ntry { value.flush(); } catch (IOException e) {\n    throw new FileConnectorException(CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n        \"Flush data to this file [\" + key + \"] failed\", e);\n}","handlingStrategy":"try-catch","validationCode":"// before job: verify storage writable\ntry (FSDataOutputStream probe = fs.create(new Path(outputDir, \".probe\"))) { probe.write(1); probe.flush(); }","typeGuard":null,"tryCatchPattern":"try {\n    job.execute();\n} catch (FileConnectorException e) {\n    if (CommonErrorCodeDeprecated.FLUSH_DATA_FAILED.equals(e.getErrorCode())) {\n        Throwable root = e.getCause(); // inspect disk-full / connectivity\n        // free space or fix FS access, then retry\n    }\n}","preventionTips":["Ensure sufficient disk/quota on the target filesystem before large jobs","Use unique output paths per job run to avoid conflicts","Test filesystem connectivity with a probe write before submitting","Retry transient remote-storage failures with backoff"],"tags":["io","file-sink","flush-failure"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}