apache/hadoop · error · FileNotFoundException

Can not create '%s' file, because parent folder does not exi

Error message

Can not create '%s' file, because parent folder does not exist: %s

What it means

GoogleHadoopFileSystem.createNonRecursive explicitly refuses to create parent directories: it resolves the GCS parent path (UriPaths.getParentPath) and throws FileNotFoundException if getFileInfo(parent) reports it does not exist. This contrasts with create(), which implicitly creates intermediates.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopFileSystem.java:350

                this, getGcsPath(hadoopPath), fileOptions, statistics), statistics);
      },
      String.format("create(%s, %s)", hadoopPath, overwrite));
  }

  @Override
  public FSDataOutputStream createNonRecursive(
      Path hadoopPath,
      FsPermission permission,
      EnumSet<CreateFlag> flags,
      int bufferSize,
      short replication,
      long blockSize,
      Progressable progress)
      throws IOException {
    URI gcsPath = getGcsPath(checkNotNull(hadoopPath, "hadoopPath must not be null"));
    URI parentGcsPath = UriPaths.getParentPath(gcsPath);
    if (!getGcsFs().getFileInfo(parentGcsPath).exists()) {
      throw new FileNotFoundException(
          String.format(
              "Can not create '%s' file, because parent folder does not exist: %s",
              gcsPath, parentGcsPath));
    }

    return create(
        hadoopPath,
        permission,
        flags.contains(CreateFlag.OVERWRITE),
        bufferSize,
        replication,
        blockSize,
        progress);
  }

  /**
   * Appends to an existing file (optional operation). Not supported.
   *

View on GitHub (pinned to 2add963021)

Solutions

Create the parent first: fs.mkdirs(path.getParent()).
Or simply use fs.create(path, ...), which creates intermediate directories implicitly.
For strict semantics, keep createNonRecursive but pre-validate the tree and fail with your own actionable error.

Example fix

// before
fs.createNonRecursive(new Path("gs://bucket/newdir/file.txt"), FsPermission.getDefault(),
    EnumSet.of(CreateFlag.CREATE), bufferSize, replication, blockSize, progress);
// -> FileNotFoundException: parent folder does not exist

// after
if (!fs.exists(path.getParent())) {
  fs.mkdirs(path.getParent());
}
fs.createNonRecursive(path, FsPermission.getDefault(),
    EnumSet.of(CreateFlag.CREATE), bufferSize, replication, blockSize, progress);

Defensive patterns

Strategy: validation

Validate before calling

if (!fs.exists(path.getParent())) {
  fs.mkdirs(path.getParent());
}
fs.createNonRecursive(path, FsPermission.getDefault(), EnumSet.of(CreateFlag.CREATE),
    bufferSize, replication, blockSize, null);

Try / catch

catch FileNotFoundException with message contains("parent folder does not exist") - create the parent (mkdirs) and retry, or switch to fs.create() which creates parents implicitly.

Prevention

Reserve createNonRecursive for strict-structure writes and always pre-build the directory tree.
Know the contrast: create() auto-creates intermediates, createNonRecursive does not.

When it happens

Trigger: fs.createNonRecursive(new Path("gs://b/newdir/file.txt"), ...) when gs://b/newdir was never created; also when a concurrent job deleted the parent between check and create.

Common situations: Code ported from FileSystems where non-recursive create auto-mkdirs; intentional strict-mode writes (fail if structure missing) where the caller forgot to build the structure first; race with cleanup jobs removing directories.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cb663ba06545074f. Report an issue: GitHub.