apache/hadoop · error · IOException

createNonRecursive unsupported for this filesystem ${this.ge

Error message

createNonRecursive unsupported for this filesystem ${this.getClass()}

What it means

createNonRecursive(f, ...) creates a file only when the parent already exists, but the base FileSystem has no generic way to enforce that, so the default throws IOException('createNonRecursive unsupported for this filesystem <class>'). Only FileSystems that explicitly override it (e.g. the local filesystem family) support the call.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:1483

   /**
    * Opens an FSDataOutputStream at the indicated Path with write-progress
    * reporting. Same as create(), except fails if parent directory doesn't
    * already exist.
    * @param f the file name to open
    * @param permission file permission
    * @param flags {@link CreateFlag}s to use for this stream.
    * @param bufferSize the size of the buffer to be used.
    * @param replication required block replication for the file.
    * @param blockSize block size
    * @param progress the progress reporter
    * @throws IOException IO failure
    * @see #setPermission(Path, FsPermission)
    * @return output stream.
    */
    public FSDataOutputStream createNonRecursive(Path f, FsPermission permission,
        EnumSet<CreateFlag> flags, int bufferSize, short replication, long blockSize,
        Progressable progress) throws IOException {
      throw new IOException("createNonRecursive unsupported for this filesystem "
          + this.getClass());
    }

  /**
   * Creates the given Path as a brand-new zero-length file.  If
   * create fails, or if it already existed, return false.
   * <i>Important: the default implementation is not atomic</i>
   * @param f path to use for create
   * @throws IOException IO failure
   * @return if create new file success true,not false.
   */
  public boolean createNewFile(Path f) throws IOException {
    if (exists(f)) {
      return false;
    } else {
      create(f, false, getConf().getInt(IO_FILE_BUFFER_SIZE_KEY,
          IO_FILE_BUFFER_SIZE_DEFAULT)).close();
      return true;

View on GitHub (pinned to 2add963021)

Solutions

  1. Replace with fs.create(f, ...) after ensuring the parent yourself: fs.exists(parent) || fs.mkdirs(parent)
  2. Branch on the filesystem type before choosing the non-recursive API
  3. If you own the FileSystem, implement createNonRecursive

Example fix

// before
try (FSDataOutputStream out = fs.createNonRecursive(f, perm, false, buf, rep, block, null)) { }

// after
Path parent = f.getParent();
if (!fs.exists(parent)) throw new FileNotFoundException(parent.toString());
try (FSDataOutputStream out = fs.create(f, true, buf, rep, block, null)) { }
Defensive patterns

Strategy: fallback

Try / catch

try {
  out = fs.createNonRecursive(f, perm, false, buf, rep, blockSize, null);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("createNonRecursive unsupported")) {
    if (!fs.exists(f.getParent())) fs.mkdirs(f.getParent());
    out = fs.create(f, true, buf, rep, blockSize, null);
  } else {
    throw e;
  }
}

Prevention

When it happens

Trigger: Calling fs.createNonRecursive(...) on an implementation that never overrode it - various third-party connectors and wrapper FileSystems.

Common situations: Shared job code written against local/HDFS behavior then run against object stores; MapReduce's non-recursive commit paths encountering an unexpected FileSystem class.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1ab50dfbbcac0b78. Report an issue: GitHub.