apache/seatunnel · warning

Failed to clean empty transaction parent directory

Error message

Failed to clean empty transaction parent directory: {}

What it means

A WARN from AbstractWriteStrategy.cleanupEmptyDirectory (checkpoint/transaction cleanup path): deleting an empty per-transaction staging directory failed with an IOException. Written data files and checkpoint commits are unaffected; only an empty temp directory may remain under the output path.

Solutions

1. Check filesystem permissions for the SeaTunnel worker user on the output directory
2. Inspect the directory for orphaned files from failed transactions and clean manually
3. Retry the job; the leftover empty directory is harmless and can be deleted
4. Monitor storage-side errors (HDFS safe mode, S3 5xx/429) coinciding with checkpoints

// preflight: writer user needs delete rights on staging dir
// hadoop fs -test -w /output/staging && echo ok

// non-fatal; only empty dirs linger. Pattern for user-side cleanup:
// try { fs.deleteEmptyDirectory(dir); } catch (IOException e) { log + manual sweep of /output/staging }

When it happens

Trigger: On checkpoint completion the write strategy moves finished files and then removes the transaction directory via hadoopFileSystemProxy.deleteEmptyDirectory; IOException from HDFS/S3 transient errors, permissions, or the directory containing leftover files.

Common situations: Permission differences between writer subtask users and the directory owner; S3 throttling during high-parallelism checkpoints; crashed previous runs leaving orphan files so the directory is not empty.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• delete transaction directory
• Discarding pending multiline line(s) for ( )
• Failed to clean empty transaction parent directory
• Failed to delete checkpoint directory
• Failed to delete checkpoint for job

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/9e7fecd774db312c. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/AbstractWriteStrategy.java:614

    private void cleanupTransactionParentDirectories(String transactionDir) {
        Path uuidDir = new Path(transactionDir).getParent();
        if (uuidDir == null) {
            return;
        }
        Path jobDir = uuidDir.getParent();
        if (jobDir == null) {
            return;
        }
        cleanupEmptyDirectory(uuidDir);
        cleanupEmptyDirectory(jobDir);
    }

    private void cleanupEmptyDirectory(Path directory) {
        try {
            hadoopFileSystemProxy.deleteEmptyDirectory(directory.toString());
        } catch (IOException e) {
            log.warn("Failed to clean empty transaction parent directory: {}", directory, e);
        }
    }

    /**
     * when a checkpoint completed, file connector should begin a new transaction and generate new
     * transaction id
     *
     * @param checkpointId checkpoint id
     */
    public void beginTransaction(Long checkpointId) {
        this.checkpointId = checkpointId;
        this.transactionId = getTransactionId(checkpointId);
        this.transactionDirectory = getTransactionDir(this.transactionId);
        this.needMoveFiles = new LinkedHashMap<>();
        this.partitionDirAndValuesMap = new LinkedHashMap<>();
    }

    private String getTransactionId(Long checkpointId) {

View on GitHub (pinned to cf67b549a7)