apache/flink · error · IOException

Output path '{}' could not be initialized. Canceling task...

Error message

Output path '{}' could not be initialized. Canceling task...

What it means

Thrown by FileOutputFormat.open on a LOCAL (non-distributed) filesystem when the output is non-parallel single-file mode (numTasks==1, OutputDirectoryMode.PARONLY) and fs.initOutPathLocalFS(path, writeMode, false) returns false. A false return means the path could not be prepared (e.g., target exists and writeMode is NO_OVERWRITE, or a directory exists where a file is expected).

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/FileOutputFormat.java:221

        Path p = this.outputFilePath;
        if (p == null) {
            throw new IOException("The file path is null.");
        }

        final FileSystem fs = p.getFileSystem();

        // if this is a local file system, we need to initialize the local output directory here
        if (!fs.isDistributedFS()) {

            if (numTasks == 1 && outputDirectoryMode == OutputDirectoryMode.PARONLY) {
                // output should go to a single file

                // prepare local output path. checks for write mode and removes existing files in
                // case of OVERWRITE mode
                if (!fs.initOutPathLocalFS(p, writeMode, false)) {
                    // output preparation failed! Cancel task.
                    throw new IOException(
                            "Output path '"
                                    + p.toString()
                                    + "' could not be initialized. Canceling task...");
                }
            } else {
                // numTasks > 1 || outDirMode == OutputDirectoryMode.ALWAYS

                if (!fs.initOutPathLocalFS(p, writeMode, true)) {
                    // output preparation failed! Cancel task.
                    throw new IOException(
                            "Output directory '"
                                    + p.toString()
                                    + "' could not be created. Canceling task...");
                }
            }
        }

        // Suffix the path with the parallel instance index, if needed

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Set WriteMode.OVERWRITE if reruns should replace the file: format.setWriteMode(WriteMode.OVERWRITE).
2. Delete the existing local file before running, or write to a fresh path.
3. If a directory genuinely exists at that path, choose a file path instead, or switch to OutputDirectoryMode.ALWAYS.

Example fix

// before
FileOutputFormat out = new FileOutputFormat(new Path("file:///tmp/result"));
// rerun → existing file, NO_OVERWRITE → init fails

// after
out.setWriteMode(WriteMode.OVERWRITE);

// Clear or allow overwrite of an existing local target
Path localOut = new Path("file:///tmp/result");
if (localOut.getFileSystem().exists(localOut)) {
    localOut.getFileSystem().delete(localOut, false);
}
// or set overwrite
format.setWriteMode(WriteMode.OVERWRITE);

When it happens

Trigger: Writing to a local file path that already exists while WriteMode.NO_OVERWRITE is set; the target path is a directory but single-file mode expects a file; local filesystem permission to create/truncate the file is missing.

Common situations: Local/IDE runs writing to an existing output file without OVERWRITE; re-running a job whose previous output file still exists; pointing the sink at a path occupied by a directory.

Related errors

• Output directory '{}' could not be created. Canceling task..
• Output path could not be initialized.
• Output directory could not be created.
• Streaming mode not support overwrite.
• Output file path may not be null.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d7a9b3fed7449c1c. Report an issue: GitHub.