{"record":{"id":"b07c43e563f4caf8","repo":"apache/seatunnel","slug":"flush-data-failed-b07c43","errorCode":"FLUSH_DATA_FAILED","errorMessage":"Flush data to this file [%s] failed","messagePattern":"Flush data to this file \\[(.+?)\\] failed","errorType":"error_code","errorClass":"FileConnectorException","httpStatus":null,"severity":"critical","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/TextWriteStrategy.java","lineNumber":127,"sourceCode":"            if (isFirstWrite.get(filePath)) {\n                isFirstWrite.put(filePath, false);\n            } else {\n                fsDataOutputStream.write(rowDelimiter.getBytes(charset));\n            }\n            fsDataOutputStream.write(serializationSchema.serialize(safeProjectedRow(seaTunnelRow)));\n        } catch (IOException e) {\n            throw CommonError.fileOperationFailed(\"TextFile\", \"write\", filePath, e);\n        }\n    }\n\n    @Override\n    public void finishAndCloseFile() {\n        beingWrittenOutputStream.forEach(\n                (key, value) -> {\n                    try {\n                        value.flush();\n                    } catch (IOException e) {\n                        throw new FileConnectorException(\n                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n                                String.format(\"Flush data to this file [%s] failed\", key),\n                                e);\n                    } finally {\n                        try {\n                            value.close();\n                        } catch (IOException e) {\n                            log.error(\"error when close output stream {}\", key, e);\n                        }\n                    }\n                    needMoveFiles.put(key, getTargetLocation(key));\n                });\n        beingWrittenOutputStream.clear();\n        isFirstWrite.clear();\n    }\n\n    @Override\n    public FSDataOutputStream getOrCreateOutputStream(@NonNull String filePath) {","sourceCodeStart":109,"sourceCodeEnd":145,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/TextWriteStrategy.java#L109-L145","documentation":"TextWriteStrategy.finishAndCloseFile flushes every open output stream when the sink finishes/closes. If OutputStream.flush() throws IOException, it wraps it as FileConnectorException with FLUSH_DATA_FAILED naming the file; the stream is still closed in the finally block. This surfaces end-of-write I/O failures such as lost connectivity to storage or disk errors.","triggerScenarios":"Job finish/checkpoint close on a text-format file sink while flush() on a buffered output stream fails — remote FS disconnection, disk full, HDFS lease issues, or file deleted underneath the writer.","commonSituations":"S3/HDFS transient outage at job completion; NFS mount dropped; disk quota exceeded mid-write; worker node storage failure; writing to a path whose parent was removed concurrently.","solutions":["Inspect the wrapped cause IOException to identify the storage-level root cause","Check disk space/quota and storage (HDFS/S3) availability on the worker at failure time","Enable sink checkpoint/rolling so partial data is flushed periodically rather than only at close","Re-run the job after storage recovers; make the sink idempotent (overwrite/unique file names) for safe retries"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { strategy.finishAndCloseFile(); } catch (FileConnectorException e) { if (e.getErrorCode() == FLUSH_DATA_FAILED) { log(e.getCause()); /* inspect storage then re-run */ } }","preventionTips":["Monitor storage health (disk, HDFS, S3) during job runtime","Use checkpointing/rolling so failures at close lose less data","Make sink output idempotent for safe re-runs"],"tags":["text","flush","io","file-sink"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}