{"record":{"id":"e083eb6b607e5eda","repo":"apache/seatunnel","slug":"flush-data-failed-e083eb","errorCode":"FLUSH_DATA_FAILED","errorMessage":"Flush data to this file [%s] failed","messagePattern":"Flush data to this file \\[(.+?)\\] failed","errorType":"error_code","errorClass":"FileConnectorException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/DebeziumJsonWriteStrategy.java","lineNumber":107,"sourceCode":"            if (isFirstWrite.get(filePath)) {\n                isFirstWrite.put(filePath, false);\n            } else {\n                fsDataOutputStream.write(rowDelimiter);\n            }\n            fsDataOutputStream.write(rowBytes);\n        } catch (IOException e) {\n            throw CommonError.fileOperationFailed(\"DebeziumJsonFile\", \"write\", filePath, e);\n        }\n    }\n\n    @Override\n    public void finishAndCloseFile() {\n        beingWrittenOutputStream.forEach(\n                (key, value) -> {\n                    try {\n                        value.flush();\n                    } catch (IOException e) {\n                        throw new FileConnectorException(\n                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n                                String.format(\"Flush data to this file [%s] failed\", key),\n                                e);\n                    } finally {\n                        try {\n                            value.close();\n                        } catch (IOException e) {\n                            log.warn(\"Close file output stream {} failed\", key, e);\n                        }\n                    }\n                    needMoveFiles.put(key, getTargetLocation(key));\n                });\n        beingWrittenOutputStream.clear();\n        isFirstWrite.clear();\n    }\n\n    @Override\n    public FSDataOutputStream getOrCreateOutputStream(@NonNull String filePath) {","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/DebeziumJsonWriteStrategy.java#L89-L125","documentation":"Thrown by DebeziumJsonWriteStrategy.finishAndCloseFile when flushing the output stream of a Debezium-JSON formatted file fails with an IOException during finalization. The buffered JSON records could not be committed to storage, so the produced file may be truncated. The underlying IOException is preserved as the cause.","triggerScenarios":"value.flush() on FSDataOutputStream in beingWrittenOutputStream throws IOException at sink close/checkpoint — disk full, remote storage (HDFS/S3) unreachable, or stream already closed.","commonSituations":"HDFS DataNode failure or replication timeout during job finish; S3/OSS throttling or expired credentials; disk quota exceeded on local checkpoint dir; concurrent writers to the same path.","solutions":["Inspect the chained cause to identify the storage-level failure and fix it (space, credentials, connectivity)","Validate the target filesystem is writable before submitting the job (write/read a probe file)","Avoid concurrent jobs writing the same output directory; use distinct transaction paths per job","Retry the pipeline; transient storage failures during finalization frequently succeed on rerun"],"exampleFix":"// before\nvalue.flush(); // unchecked IOException propagates without context\n// after\ntry { value.flush(); } catch (IOException e) {\n    throw new FileConnectorException(CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n        String.format(\"Flush data to this file [%s] failed\", key), e);\n}","handlingStrategy":"try-catch","validationCode":"// pre-check filesystem writability\nassert fs.createNewFile(new Path(outputDir, \".write_check\"));","typeGuard":null,"tryCatchPattern":"try {\n    job.execute();\n} catch (FileConnectorException e) {\n    if (CommonErrorCodeDeprecated.FLUSH_DATA_FAILED.equals(e.getErrorCode())) {\n        log.error(\"Flush failed: \", e.getCause());\n        // remediate storage then resubmit\n    }\n}","preventionTips":["Monitor storage quota/credentials validity for long-running jobs","Avoid concurrent writers to the same output directory","Use job-unique transaction/temp directories","Catch FileConnectorException and inspect getCause() for root cause"],"tags":["io","file-sink","flush-failure"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}