What causes "Failed to mkdir on {}" in apache/hadoop?

Create and probe the parent as the same user: `mkdir -p "$(dirname )" && touch .probe && rm .probe`

apache/hadoop · error · IOException

Failed to mkdir on {}

Error message

Failed to mkdir on {}

What it means

Same LevelDB temp-dir setup, next failure mode: File.mkdirs() on the -t path returned false, so the directory could not be created at all. This is an OS-level problem — a missing or unwritable parent, a regular file shadowing a path component, a read-only filesystem, or an SELinux/ACL denial.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java:400

      protected boolean removeEldestEntry(Map.Entry<Long, String> entry) {
        return super.size() > CAPACITY;
      }
    }

    /** Map the child inode to the parent directory inode. */
    private LevelDBStore dirChildMap = null;
    /** Directory entry map */
    private LevelDBStore dirMap = null;
    private DirPathCache dirPathCache = new DirPathCache();

    LevelDBMetadataMap(String baseDir) throws IOException {
      File dbDir = new File(baseDir);
      if (dbDir.exists()) {
        throw new IOException("Folder " + dbDir + " already exists! Delete " +
            "manually or provide another (not existing) directory!");
      }
      if (!dbDir.mkdirs()) {
        throw new IOException("Failed to mkdir on " + dbDir);
      }
      try {
        dirChildMap = new LevelDBStore(new File(dbDir, "dirChildMap"));
        dirMap = new LevelDBStore(new File(dbDir, "dirMap"));
      } catch (IOException e) {
        LOG.error("Failed to open LevelDBs", e);
        IOUtils.cleanupWithLogger(null, this);
      }
    }

    @Override
    public void close() throws IOException {
      IOUtils.cleanupWithLogger(null, dirChildMap, dirMap);
      dirChildMap = null;
      dirMap = null;
    }

    private static byte[] toBytes(long value) {

View on GitHub (pinned to 2add963021)

Solutions

Create and probe the parent as the same user: `mkdir -p "$(dirname <dir>)" && touch <dir>.probe && rm <dir>.probe`
Move -t to a location the running user owns (home directory, /tmp/$USER)
Check each path component with `ls -l` for a file masquerading as a directory, and `mount` for a read-only filesystem

Example fix

# before: parent not writable
hdfs oiv -p delimited -i fsimage -o out.csv -t /var/lib/hadoop/oivtmp
# IOException: Failed to mkdir on /var/lib/hadoop/oivtmp

# after: writable parent, created first
mkdir -p ~/oiv-parent
hdfs oiv -p delimited -i fsimage -o out.csv -t ~/oiv-parent/run1

Defensive patterns

Strategy: validation

Validate before calling

File dbDir = new File(tempPath);
File parent = dbDir.getAbsoluteFile().getParentFile();
if (dbDir.exists()) {
  throw new IllegalStateException("temp dir " + dbDir + " already exists");
}
if (parent == null || !parent.isDirectory() || !parent.canWrite()) {
  throw new IllegalStateException(
      "cannot create " + dbDir + ": parent missing, not a directory, or not writable");
}

Prevention

Create the parent directory yourself (mkdir -p) before running oiv with -t
Point -t at storage the oiv user owns (home dir, /tmp/$USER)
Watch for regular files shadowing path components and read-only mounts

When it happens

Trigger: Pointing -t at a path whose parent does not exist or is not writable by the oiv user; a path component is a file (e.g. -t /tmp/x/y where /tmp/x is a regular file); read-only or SELinux-confined mount.

Common situations: Running oiv as an unprivileged user against system paths; NFS or read-only mounts; typos in the temp path.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8b9aaef6ee9fe4cb. Report an issue: GitHub.