{"record":{"id":"9a2678c91318b7a4","repo":"apache/dolphinscheduler","slug":"failed-to-create-parent-directory-for-destination","errorCode":null,"errorMessage":"Failed to create parent directory for destination file","messagePattern":"Failed to create parent directory for destination file","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"dolphinscheduler-storage-plugin/dolphinscheduler-storage-hdfs/src/main/java/org/apache/dolphinscheduler/plugin/storage/hdfs/HdfsStorageOperator.java","lineNumber":188,"sourceCode":"\n    @SneakyThrows\n    @Override\n    public void download(String srcHdfsFilePath, String dstFile, boolean overwrite) {\n        Path srcPath = new Path(srcHdfsFilePath);\n        File dstPath = new File(dstFile);\n\n        if (dstPath.exists()) {\n            if (dstPath.isFile()) {\n                if (overwrite) {\n                    Files.delete(dstPath.toPath());\n                }\n            } else {\n                log.error(\"destination file must be a file\");\n            }\n        }\n\n        if (!dstPath.getParentFile().exists() && !dstPath.getParentFile().mkdirs()) {\n            throw new IOException(\"Failed to create parent directory for destination file\");\n        }\n\n        FileUtil.copy(fs, srcPath, dstPath, false, fs.getConf());\n    }\n\n    @SneakyThrows\n    @Override\n    public void copy(String srcPath, String dstPath, boolean deleteSource, boolean overwrite) {\n        FileUtil.copy(fs, new Path(srcPath), fs, new Path(dstPath), deleteSource, overwrite, fs.getConf());\n    }\n\n    @SneakyThrows\n    @Override\n    public void upload(String srcAbsoluteFilePath,\n                       String dstAbsoluteFilePath,\n                       boolean deleteSource,\n                       boolean overwrite) {\n        Path srcPath = new Path(srcAbsoluteFilePath);","sourceCodeStart":170,"sourceCodeEnd":206,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-storage-plugin/dolphinscheduler-storage-hdfs/src/main/java/org/apache/dolphinscheduler/plugin/storage/hdfs/HdfsStorageOperator.java#L170-L206","documentation":"HdfsStorageOperator.download copies a file from HDFS to a local destination. Before copying it ensures the destination's parent directory exists; if the parent is missing and mkdirs() fails it throws IOException, aborting the download.","triggerScenarios":"Calling download(srcHdfsPath, dstLocalPath) where dstLocalPath's parent directory does not exist and File.mkdirs() fails — typically due to filesystem permissions, a read-only filesystem, or a parent path component being an existing regular file.","commonSituations":"Downloading into a temp dir whose permissions changed; destination parent path conflicting with an existing file; disk mounted read-only or full.","solutions":["Pre-create the parent directory manually (mkdir -p) with correct permissions and retry","Check that no regular file exists at a path component of the destination parent","Verify write permission and disk space on the destination filesystem"],"exampleFix":"// before\nhdfs.download(hdfsPath, \"/data/locked/out/result.csv\");\n// after\nFiles.createDirectories(Paths.get(\"/data/out\"));\nhdfs.download(hdfsPath, \"/data/out/result.csv\");","handlingStrategy":"try-catch","validationCode":"File parent = new File(dstLocalPath).getParentFile();\nif (!parent.exists() && !parent.mkdirs()) throw new IOException(\"Cannot create \" + parent);\nif (parent.exists() && !parent.canWrite()) throw new IOException(\"No write permission: \" + parent);","typeGuard":null,"tryCatchPattern":"try {\n    hdfs.download(src, dst);\n} catch (IOException e) {\n    if (e.getMessage().contains(\"Failed to create parent directory\")) {\n        Files.createDirectories(Paths.get(dst).getParent());\n        hdfs.download(src, dst); // retry once\n    } else throw e;\n}","preventionTips":["Pre-create destination directories with correct permissions","Ensure no regular file blocks a parent path component","Check disk space and mount status (not read-only) before downloads"],"tags":["hdfs","io","filesystem"],"backgroundTag":"mkdir-permission-denied","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}