{"record":{"id":"b0f42e21f026402c","repo":"apache/seatunnel","slug":"flush-data-failed","errorCode":"FLUSH_DATA_FAILED","errorMessage":"Flush data to this file [%s] failed","messagePattern":"Flush data to this file \\[(.+?)\\] failed","errorType":"error_code","errorClass":"FileConnectorException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/BinaryWriteStrategy.java","lineNumber":161,"sourceCode":"        return fsDataOutputStream;\n    }\n\n    @Override\n    public void applySchemaChange(SchemaChangeEvent event) {\n        throw new FileConnectorException(\n                FileConnectorErrorCode.FORMAT_NOT_SUPPORT,\n                \"BinaryWriteStrategy does not support schema evolution. \"\n                        + \"Binary format requires a fixed schema.\");\n    }\n\n    @Override\n    public void finishAndCloseFile() {\n        beingWrittenOutputStream.forEach(\n                (key, value) -> {\n                    try {\n                        value.flush();\n                    } catch (IOException e) {\n                        throw new FileConnectorException(\n                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n                                String.format(\"Flush data to this file [%s] failed\", key),\n                                e);\n                    } finally {\n                        try {\n                            value.close();\n                        } catch (IOException e) {\n                            log.error(\"error when close output stream {}\", key, e);\n                        }\n                    }\n                    needMoveFiles.put(key, getTargetLocation(key));\n                });\n        beingWrittenOutputStream.clear();\n        partIndexMap.clear();\n    }\n}\n","sourceCodeStart":143,"sourceCodeEnd":178,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/BinaryWriteStrategy.java#L143-L178","documentation":"At the end of a sink task, finishAndCloseFile flushes and closes every open output stream. If flush() throws IOException for a given file, BinaryWriteStrategy wraps it in a FileConnectorException with code FLUSH_DATA_FAILED naming the offending file. Data buffered for that file was not durably written, so the task fails rather than silently losing bytes.","triggerScenarios":"finishAndCloseFile iterates beingWrittenOutputStream and the underlying Hadoop FSDataOutputStream.flush() throws — storage endpoint outage, disk full, HDFS/S3 connection dropped mid-write.","commonSituations":"S3/HDFS network interruption at task completion; quota or disk-space exhaustion on the target filesystem; stale file handles after a long-running task.","solutions":["Inspect the wrapped IOException cause for the storage-level root cause (connectivity, space, quota)","Verify free space/quotas on the target filesystem and retry the job","Add storage retry/timeout tuning (e.g. S3 client retries, HDFS client timeouts) for flaky endpoints"],"exampleFix":"null","handlingStrategy":"try-catch","validationCode":"// pre-flight: ensure the target filesystem accepts writes and has space\ntry (FSDataOutputStream probe = fs.create(new Path(sinkPath, \".probe\"))) {\n  probe.write(new byte[1]); probe.hflush();\n  fs.delete(new Path(sinkPath, \".probe\"), true);\n}","typeGuard":null,"tryCatchPattern":"try { strategy.finishAndCloseFile(); } catch (FileConnectorException e) {\n  if (CommonErrorCode.FLUSH_DATA_FAILED.equals(e.getErrorCode())) {\n    log.error(\"Flush failed for output file; storage may be unavailable: {}\", e.getCause());\n    // rely on engine retry/checkpoint to re-run the task\n  }\n}","preventionTips":["Monitor disk space and object-store quotas on sink workers","Configure storage client retries/timeouts for flaky S3/HDFS endpoints","Keep checkpoint intervals moderate so buffers are flushed regularly"],"tags":["file-sink","binary","flush","io"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}