{"record":{"id":"d014cfcaa73dc48d","repo":"apache/hadoop","slug":"cannot-create-directories-because-of-existing-file","errorCode":null,"errorMessage":"Cannot create directories because of existing file: %s","messagePattern":"Cannot create directories because of existing file: (.+?)","errorType":"exception","errorClass":"FileAlreadyExistsException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageFileSystem.java","lineNumber":695,"sourceCode":"    // Create a list of all files that can conflict with intermediate/subdirectory paths.\n    // For example: gs://foo/bar/zoo/ => (gs://foo/bar, gs://foo/bar/zoo)\n    List<StorageResourceId> fileIds =\n        getDirs(resourceId.getObjectName()).stream()\n            .filter(subdir -> !isNullOrEmpty(subdir))\n            .map(\n                subdir ->\n                    new StorageResourceId(\n                        resourceId.getBucketName(), StringPaths.toFilePath(subdir)))\n            .collect(toImmutableList());\n\n    // Each intermediate path must ensure that corresponding file does not exist\n    //\n    // If for any of the intermediate paths file already exists then bail out early.\n    // It is possible that the status of intermediate paths can change after\n    // we make this check therefore this is a good faith effort and not a guarantee.\n    for (GoogleCloudStorageItemInfo fileInfo : gcs.getItemInfos(fileIds)) {\n      if (fileInfo.exists()) {\n        throw new FileAlreadyExistsException(\n            \"Cannot create directories because of existing file: \" + fileInfo.getResourceId());\n      }\n    }\n  }\n\n  /**\n   * For objects whose name looks like a path (foo/bar/zoo), returns all directory paths.\n   *\n   * <p>For example:\n   *\n   * <ul>\n   *   <li>foo/bar/zoo => returns: (foo/, foo/bar/)\n   *   <li>foo/bar/zoo/ => returns: (foo/, foo/bar/, foo/bar/zoo/)\n   *   <li>foo => returns: ()\n   * </ul>\n   *\n   * @param objectName Name of an object.\n   * @return List of subdirectory like paths.","sourceCodeStart":677,"sourceCodeEnd":713,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageFileSystem.java#L677-L713","documentation":"GoogleCloudStorageFileSystem.mkdir checks every intermediate directory path of the requested tree via gcs.getItemInfos; if any intermediate path exists as a FILE (a storage object occupying that name), it throws FileAlreadyExistsException naming the offending StorageResourceId. GCS's flat namespace means an object named a/b permanently blocks using a/b as a directory.","triggerScenarios":"fs.mkdirs(new Path(\"gs://b/dir/sub/deeper\")) when an object named gs://b/dir/sub exists as a file. Any mkdirs whose path goes 'through' an existing file.","commonSituations":"ETL layouts where the same name is used as a file at one stage and a directory at another; leftover marker/empty files from previous runs occupying directory names; data migrated from HDFS with name collisions.","solutions":["Delete the conflicting file (its StorageResourceId is printed in the exception message) if it is safe to remove.","Rename the conflicting file out of the way, then retry mkdirs.","Choose a different directory name that does not traverse an existing file.","Catch FileAlreadyExistsException and surface which intermediate path collided."],"exampleFix":"// before\nfs.mkdirs(new Path(\"gs://bucket/dir/file.txt/sub\"));\n// -> FileAlreadyExistsException: existing file: gs://bucket/dir/file.txt\n\n// after\nPath conflicting = new Path(\"gs://bucket/dir/file.txt\");\nif (fs.exists(conflicting) && !fs.getFileStatus(conflicting).isDirectory()) {\n  fs.delete(conflicting, false);\n}\nfs.mkdirs(new Path(\"gs://bucket/dir/file.txt/sub\"));","handlingStrategy":"try-catch","validationCode":"// optional pre-check of ancestors\nPath p = new Path(\"gs://bucket/a/b/c/\");\nPath anc = p.getParent();\nwhile (anc != null && !anc.isRoot()) {\n  if (fs.exists(anc) && !fs.getFileStatus(anc).isDirectory()) {\n    throw new IllegalStateException(\"Ancestor is a file: \" + anc);\n  }\n  anc = anc.getParent();\n}","typeGuard":null,"tryCatchPattern":"try {\n  fs.mkdirs(path);\n} catch (FileAlreadyExistsException e) {\n  // e.getMessage() names the colliding StorageResourceId; delete/rename that file or pick another directory name\n}","preventionTips":["Keep file names and directory names disjoint in layout schemas (a file at a/b forever blocks directory a/b/...).","Catch FileAlreadyExistsException specifically rather than IOException to surface the colliding resource id.","Clean leftover marker files from earlier runs before mkdirs stages."],"tags":["mkdir","gcs","hadoop","directories","file-conflict"],"backgroundTag":"mkdir-file-conflict","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}