What causes "Failed to close failure-data writer. cause={}" in apache/seatunnel?

Check disk space and mount health on the worker node at job shutdown

Failed to close failure-data writer. cause={}

Error message

Failed to close failure-data writer. cause={}

What it means

Logged by BatchBuffer.closeFailureWriter during task close() when closing the BufferedWriter for the failure-data file throws an IOException. The reference is nulled in a finally block so no leak persists; the warning only means buffered failure samples may not have been fully flushed to disk at close time.

Source

Thrown at seatunnel-connectors-v2/connector-hugegraph/src/main/java/org/apache/seatunnel/connectors/seatunnel/hugegraph/buffer/BatchBuffer.java:440

            }
            closed = true;
        }

        LOG.info("Closing BatchBuffer, performing final flush...");
        try {
            flush();
        } finally {
            closeFailureWriter();
        }
        LOG.info("BatchBuffer closed.");
    }

    private void closeFailureWriter() {
        if (failureWriter != null) {
            try {
                failureWriter.close();
            } catch (IOException e) {
                LOG.warn("Failed to close failure-data writer. cause={}", e.getMessage());
            } finally {
                failureWriter = null;
            }
        }
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

Check disk space and mount health on the worker node at job shutdown
Review the failure-data file for truncated final lines; treat the last sample as possibly incomplete
Ensure the failureDataPath points to reliable local storage rather than a flaky network mount
This is logged as WARN and does not fail the job; re-run data validation if failure samples matter for auditing

Defensive patterns

Strategy: try-catch

Try / catch

// The connector already swallows this; treat it as advisory:
// after close(), optionally verify the failure file is complete
if (!failureFileComplete(failureDataPath)) {
    LOG.warn("failure samples may be truncated due to close error");
}

Prevention

Use reliable local storage (not flaky network mounts) for failureDataPath
Keep adequate disk headroom so close-time flush cannot fail from disk-full
Avoid external cleanup cron jobs that delete files under running jobs
Validate failure-sample contents after jobs that logged this warning

When it happens

Trigger: failureWriter.close() throws IOException during the sink's close lifecycle: disk full while flushing remaining buffered lines, the underlying file was removed/unclosed mount, or the filesystem became unavailable before job shutdown. Called once from close().

Common situations: Disk-full on the worker during shutdown, NFS/network mounts dropped mid-job, or container shutdown racing with the flush of the failure-sample file.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/c8698270faf3a7f6. Report an issue: GitHub.