{"record":{"id":"ce8590a3386349c8","repo":"apache/seatunnel","slug":"close-file-output-stream-failed-ce8590","errorCode":null,"errorMessage":"Close file output stream {} failed","messagePattern":"Close file output stream (.+?) failed","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/MaxWellJsonWriteStrategy.java","lineNumber":115,"sourceCode":"        }\n    }\n\n    @Override\n    public void finishAndCloseFile() {\n        beingWrittenOutputStream.forEach(\n                (key, value) -> {\n                    try {\n                        value.flush();\n                    } catch (IOException e) {\n                        throw new FileConnectorException(\n                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,\n                                String.format(\"Flush data to this file [%s] failed\", key),\n                                e);\n                    } finally {\n                        try {\n                            value.close();\n                        } catch (IOException e) {\n                            log.warn(\"Close file output stream {} failed\", key, e);\n                        }\n                    }\n                    needMoveFiles.put(key, getTargetLocation(key));\n                });\n        beingWrittenOutputStream.clear();\n        isFirstWrite.clear();\n    }\n\n    @Override\n    public FSDataOutputStream getOrCreateOutputStream(@NonNull String filePath) {\n        FSDataOutputStream fsDataOutputStream = beingWrittenOutputStream.get(filePath);\n        if (fsDataOutputStream == null) {\n            try {\n                switch (compressFormat) {\n                    case LZO:\n                        LzopCodec lzo = new LzopCodec();\n                        OutputStream out =\n                                lzo.createOutputStream(","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/MaxWellJsonWriteStrategy.java#L97-L133","documentation":"Logged in MaxWellJsonWriteStrategy.finishAndCloseFile when closing the output stream of a finished MaxWell JSON file throws an IOException. Data flush errors are reported separately; this warning covers only the close step. The file still gets registered for move to its target location.","triggerScenarios":"At sink finish/commit, finishAndCloseFile calls value.close() on the per-file stream; an IOException from the underlying Hadoop FileSystem (lost DataNode connection, S3 finalize error, deleted staging file) triggers this log.","commonSituations":"Network instability at job end; object-store transient failures; Kerberos/credentials expiring on long jobs; external cleanup removing files mid-commit.","solutions":["Inspect the logged exception cause and resolve the storage or network problem","Re-run the job so checkpoint/restart rewrites affected files","Verify staging directory permissions and that no external process deletes files","Check FS client retry/timeouts and backend health"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// verify storage availability before commit-sensitive jobs\nFileSystem fs = targetDir.getFileSystem(conf);\nif (!fs.exists(targetDir)) throw new IllegalStateException(\"target dir missing: \" + targetDir);","typeGuard":null,"tryCatchPattern":"try {\n    sink.close();\n} catch (Exception e) {\n    log.warn(\"Close failed; will rely on job retry/verification\", e);\n    // confirm files moved to target location and are non-empty\n}","preventionTips":["Ensure stable network to storage at job completion time","Disable concurrent jobs/processes touching the same staging files","Renew or extend credentials for long-running jobs","Post-job checks: list target files and compare with expected partitions"],"tags":["ioexception","hadoop","file-close","cdc"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}