apache/hadoop · error · IOException

Mkdirs failed to create ${badDir}

Error message

Mkdirs failed to create ${badDir}

What it means

In the same bad-file quarantine flow, LocalFileSystem.reportChecksumFailure() creates <highest-writable-parent>/bad_files via mkdirs(); if mkdirs() returns false and the path is not already a directory, this IOException aborts moving the corrupt file (and its checksum) aside.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java:132

      String device = new DF(f, getConf()).getMount();
      File parent = f.getParentFile();
      File dir = null;
      while (parent != null && FileUtil.canWrite(parent) &&
          parent.toString().startsWith(device)) {
        dir = parent;
        parent = parent.getParentFile();
      }

      if (dir==null) {
        throw new IOException(
                              "not able to find the highest writable parent dir");
      }
        
      // move the file there
      File badDir = new File(dir, "bad_files");
      if (!badDir.mkdirs()) {
        if (!badDir.isDirectory()) {
          throw new IOException("Mkdirs failed to create " + badDir.toString());
        }
      }
      String suffix = "." + rand.nextInt();
      File badFile = new File(badDir, f.getName()+suffix);
      LOG.warn("Moving bad file " + f + " to " + badFile);
      in.close();                               // close it first
      boolean b = f.renameTo(badFile);                      // rename it
      if (!b) {
        LOG.warn("Ignoring failure of renameTo");
      }
      // move checksum file too
      File checkFile = ((RawLocalFileSystem)fs).pathToFile(getChecksumFile(p));
      // close the stream before rename to release the file handle
      sums.close();
      b = checkFile.renameTo(new File(badDir, checkFile.getName()+suffix));
      if (!b) {
          LOG.warn("Ignoring failure of renameTo");
        }

View on GitHub (pinned to 2add963021)

Solutions

  1. Free space or repair the disk that holds the bad_files parent
  2. chmod/chown the parent so the process user can create bad_files
  3. Remove any non-directory entry named bad_files and rerun

Example fix

// before: bad_files cannot be created under the writable parent
// after
$ sudo -u <procuser> mkdir -p <writable-parent>/bad_files
$ sudo chown <procuser> <writable-parent>/bad_files
Defensive patterns

Strategy: try-catch

Validate before calling

File bad = new File(writableParent, "bad_files");
if (!bad.isDirectory() && !bad.mkdirs()) {
  throw new IOException("cannot create quarantine dir " + bad);
}

Try / catch

try {
  in = localFs.open(p);
} catch (IOException e) {
  // 'Mkdirs failed to create ...bad_files': free disk space / fix ownership, then re-read
}

Prevention

When it happens

Trigger: reportChecksumFailure() quarantine when bad_files cannot be created: parent permissions changed between the canWrite scan and mkdirs, disk full, volume remounted read-only, or a non-directory file already occupies the bad_files name.

Common situations: Full disks, security hardening stripping write bits at runtime, leftover file named bad_files from earlier failed runs.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/36f58eb9d080d5fa. Report an issue: GitHub.