{"record":{"id":"a65cd5048f119b5d","repo":"apache/flink","slug":"output-directory-could-not-be-created-cancel","errorCode":null,"errorMessage":"Output directory '{}' could not be created. Canceling task...","messagePattern":"Output directory '(.+?)' could not be created\\. Canceling task\\.\\.\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/common/io/FileOutputFormat.java","lineNumber":231,"sourceCode":"\n            if (numTasks == 1 && outputDirectoryMode == OutputDirectoryMode.PARONLY) {\n                // output should go to a single file\n\n                // prepare local output path. checks for write mode and removes existing files in\n                // case of OVERWRITE mode\n                if (!fs.initOutPathLocalFS(p, writeMode, false)) {\n                    // output preparation failed! Cancel task.\n                    throw new IOException(\n                            \"Output path '\"\n                                    + p.toString()\n                                    + \"' could not be initialized. Canceling task...\");\n                }\n            } else {\n                // numTasks > 1 || outDirMode == OutputDirectoryMode.ALWAYS\n\n                if (!fs.initOutPathLocalFS(p, writeMode, true)) {\n                    // output preparation failed! Cancel task.\n                    throw new IOException(\n                            \"Output directory '\"\n                                    + p.toString()\n                                    + \"' could not be created. Canceling task...\");\n                }\n            }\n        }\n\n        // Suffix the path with the parallel instance index, if needed\n        this.actualFilePath =\n                (numTasks > 1 || outputDirectoryMode == OutputDirectoryMode.ALWAYS)\n                        ? p.suffix(\"/\" + getDirectoryFileName(taskNumber))\n                        : p;\n\n        // create output file\n        this.stream = fs.create(this.actualFilePath, writeMode);\n\n        // at this point, the file creation must have succeeded, or an exception has been thrown\n        this.fileCreated = true;","sourceCodeStart":213,"sourceCodeEnd":249,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/common/io/FileOutputFormat.java#L213-L249","documentation":"Thrown by FileOutputFormat.open on a LOCAL filesystem when output is parallel (numTasks>1) or OutputDirectoryMode.ALWAYS, and fs.initOutPathLocalFS(path, writeMode, true) returns false. The true flag requests directory creation; failure means the output directory could not be created or cleared (existing non-directory target, permissions, parent missing).","triggerScenarios":"A file (not a directory) exists at the target path so directory creation fails; parent directories are missing and not auto-created; local permissions deny directory creation; NO_OVERWRITE against an existing directory with conflicting contents.","commonSituations":"Parallel local sink where a previous run left a single file at the directory path; pointing the parallel sink at a path occupied by a regular file; restricted local filesystem permissions in a container.","solutions":["Remove any existing file at the target path, or use WriteMode.OVERWRITE.","Ensure the parent directory exists and is writable, or let the FS auto-create it with proper config.","If a single file already exists there, switch OutputDirectoryMode or clear it first."],"exampleFix":"// before: parallel sink, prior file at path\nFileOutputFormat out = new FileOutputFormat(new Path(\"file:///tmp/out\"));\nenv.setParallelism(4);\n\n// after\nout.setWriteMode(WriteMode.OVERWRITE);","handlingStrategy":"validation","validationCode":"// Ensure the target is clear for a parallel local directory write\nPath dir = new Path(\"file:///tmp/out\");\nFileSystem fs = dir.getFileSystem();\nif (fs.exists(dir) && !fs.getFileStatus(dir).isDir()) {\n    fs.delete(dir, false); // remove stray file\n}\nformat.setWriteMode(WriteMode.OVERWRITE);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use WriteMode.OVERWRITE for parallel local sinks.","Remove any stray file occupying the directory path.","Ensure the parent directory is writable in containers."],"tags":["flink","file-output","local-fs","directory","write-mode"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}