{"record":{"id":"cdbd420394339c6d","repo":"apache/seatunnel","slug":"flush-data-failed-cdbd42","errorCode":"FLUSH_DATA_FAILED","errorMessage":"Flush data to this file [%s] failed","messagePattern":"Flush data to this file \\[(.+?)\\] failed","errorType":"error_code","errorClass":"FileConnectorException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/MaxWellJsonWriteStrategy.java","lineNumber":107,"sourceCode":"            if (isFirstWrite.get(filePath)) {\n                isFirstWrite.put(filePath, false);\n            } else {\n                fsDataOutputStream.write(rowDelimiter);\n            }\n            fsDataOutputStream.write(rowBytes);\n        } catch (IOException e) {\n            throw CommonError.fileOperationFailed(\"MaxWellJsonFile\", \"write\", filePath, e);\n        }\n    }\n\n    @Override\n    public void finishAndCloseFile() {\n        beingWrittenOutputStream.forEach(\n                (key, value) -> {\n                    try {\n                        value.flush();\n                    } catch (IOException e) {\n                        throw new FileConnectorException(\n                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n                                String.format(\"Flush data to this file [%s] failed\", key),\n                                e);\n                    } finally {\n                        try {\n                            value.close();\n                        } catch (IOException e) {\n                            log.warn(\"Close file output stream {} failed\", key, e);\n                        }\n                    }\n                    needMoveFiles.put(key, getTargetLocation(key));\n                });\n        beingWrittenOutputStream.clear();\n        isFirstWrite.clear();\n    }\n\n    @Override\n    public FSDataOutputStream getOrCreateOutputStream(@NonNull String filePath) {","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/MaxWellJsonWriteStrategy.java#L89-L125","documentation":"Thrown by MaxWellJsonWriteStrategy.finishAndCloseFile when flushing the output stream of a Maxwell-JSON formatted file fails with an IOException during file finalization. Data buffered in memory could not be persisted, so the output file may be truncated. The triggering IOException is chained as the cause.","triggerScenarios":"value.flush() on the FSDataOutputStream in beingWrittenOutputStream throws IOException at sink close — disk full, HDFS/S3 outage, stream already closed, or file removed externally.","commonSituations":"Object-store throttling or credential expiry near the end of a long job; HDFS safe mode or DataNode loss at finalize time; local disk quota exceeded; concurrent writers on same path.","solutions":["Inspect the chained cause and fix the storage-level issue (space, permissions, connectivity)","Verify the target filesystem is healthy and writable before the job finishes (probe write)","Guarantee exclusive write access to output paths; isolate each job's transaction directory","Retry the job after remediation — finalize-time flush failures are often transient"],"exampleFix":"// before\nvalue.flush(); // loses error context\n// after\ntry { value.flush(); } catch (IOException e) {\n    throw new FileConnectorException(CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n        String.format(\"Flush data to this file [%s] failed\", key), e);\n}","handlingStrategy":"try-catch","validationCode":"// sanity-check target storage before submit\nfs.access(outDir, FsAction.WRITE); // throws if not writable","typeGuard":null,"tryCatchPattern":"try {\n    job.execute();\n} catch (FileConnectorException e) {\n    if (CommonErrorCodeDeprecated.FLUSH_DATA_FAILED.equals(e.getErrorCode())) {\n        Throwable cause = e.getCause();\n        // e.g. refresh credentials if cause is credential/token related, then retry\n    }\n}","preventionTips":["Refresh object-store credentials before long jobs finish","Validate the target filesystem is healthy (safe mode, datanodes) before finalize","Isolate each job's output in its own directory","Inspect chained cause for actionable storage errors"],"tags":["io","file-sink","flush-failure"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}