What causes "Close file output stream {} failed" in apache/seatunnel?

Inspect the logged exception cause and resolve the storage or network problem

Close file output stream

Error message

Close file output stream {} failed

What it means

Logged in MaxWellJsonWriteStrategy.finishAndCloseFile when closing the output stream of a finished MaxWell JSON file throws an IOException. Data flush errors are reported separately; this warning covers only the close step. The file still gets registered for move to its target location.

Solutions

Inspect the logged exception cause and resolve the storage or network problem
Re-run the job so checkpoint/restart rewrites affected files
Verify staging directory permissions and that no external process deletes files
Check FS client retry/timeouts and backend health

Defensive patterns

Strategy: retry

Validate before calling

// verify storage availability before commit-sensitive jobs
FileSystem fs = targetDir.getFileSystem(conf);
if (!fs.exists(targetDir)) throw new IllegalStateException("target dir missing: " + targetDir);

Try / catch

try {
    sink.close();
} catch (Exception e) {
    log.warn("Close failed; will rely on job retry/verification", e);
    // confirm files moved to target location and are non-empty
}

Prevention

Ensure stable network to storage at job completion time
Disable concurrent jobs/processes touching the same staging files
Renew or extend credentials for long-running jobs
Post-job checks: list target files and compare with expected partitions

When it happens

Trigger: At sink finish/commit, finishAndCloseFile calls value.close() on the per-file stream; an IOException from the underlying Hadoop FileSystem (lost DataNode connection, S3 finalize error, deleted staging file) triggers this log.

Common situations: Network instability at job end; object-store transient failures; Kerberos/credentials expiring on long jobs; external cleanup removing files mid-commit.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/ce8590a3386349c8. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/MaxWellJsonWriteStrategy.java:115

        }
    }

    @Override
    public void finishAndCloseFile() {
        beingWrittenOutputStream.forEach(
                (key, value) -> {
                    try {
                        value.flush();
                    } catch (IOException e) {
                        throw new FileConnectorException(
                                CommonErrorCodeDeprecated.FLUSH_DATA_FAILED,
                                String.format("Flush data to this file [%s] failed", key),
                                e);
                    } finally {
                        try {
                            value.close();
                        } catch (IOException e) {
                            log.warn("Close file output stream {} failed", key, e);
                        }
                    }
                    needMoveFiles.put(key, getTargetLocation(key));
                });
        beingWrittenOutputStream.clear();
        isFirstWrite.clear();
    }

    @Override
    public FSDataOutputStream getOrCreateOutputStream(@NonNull String filePath) {
        FSDataOutputStream fsDataOutputStream = beingWrittenOutputStream.get(filePath);
        if (fsDataOutputStream == null) {
            try {
                switch (compressFormat) {
                    case LZO:
                        LzopCodec lzo = new LzopCodec();
                        OutputStream out =
                                lzo.createOutputStream(

View on GitHub (pinned to cf67b549a7)