What causes "Failed to create output dir: {}" in apache/hadoop?

Ensure the -o path does not already exist as a file; delete or rename the conflicting file, or pick another directory.

apache/hadoop · error · IOException

Failed to create output dir: {}

Error message

Failed to create output dir: {}

What it means

ImageWriter's constructor creates the output directory (opts.outdir, set by -o) before writing the generated fsimage; when outfs.mkdirs(tmp) returns false rather than throwing, it becomes IOException('Failed to create output dir: <path>'). A false return typically means the target exists as a regular file, a parent is missing or unwritable, or the URI scheme resolved to a filesystem that cannot create the path.

Source

Thrown at hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/ImageWriter.java:126

      .setLayoutVersion(LAYOUT_VERSION);

  private final String blockPoolID;

  public static Options defaults() {
    return new Options();
  }

  @SuppressWarnings("unchecked")
  public ImageWriter(Options opts) throws IOException {
    final OutputStream out;
    if (null == opts.outStream) {
      FileSystem fs = opts.outdir.getFileSystem(opts.getConf());
      outfs = (fs instanceof LocalFileSystem)
          ? ((LocalFileSystem)fs).getRaw()
          : fs;
      Path tmp = opts.outdir;
      if (!outfs.mkdirs(tmp)) {
        throw new IOException("Failed to create output dir: " + tmp);
      }
      try (NNStorage stor = new NNStorage(opts.getConf(),
          Arrays.asList(tmp.toUri()), Arrays.asList(tmp.toUri()))) {
        NamespaceInfo info = NNStorage.newNamespaceInfo();
        if (info.getLayoutVersion() != LAYOUT_VERSION) {
          throw new IllegalStateException("Incompatible layout " +
              info.getLayoutVersion() + " (expected " + LAYOUT_VERSION + ")");
        }
        // set the cluster id, if given
        if (opts.clusterID.length() > 0) {
          info.setClusterID(opts.clusterID);
        }
        // if block pool id is given
        if (opts.blockPoolID.length() > 0) {
          info.setBlockPoolID(opts.blockPoolID);
        }

        stor.format(info);

View on GitHub (pinned to 2add963021)

Solutions

Ensure the -o path does not already exist as a file; delete or rename the conflicting file, or pick another directory.
Check write permission on the parent directory for the user running fs2img.
Use an explicit, scheme-correct URI for -o (file:///abs/path for local, hdfs://ns/path for HDFS).

Example fix

# before
touch /img && hadoop fs2img -o file:///img /data

# after
rm -f /img && hadoop fs2img -o file:///img /data

Defensive patterns

Strategy: validation

Validate before calling

Path outdir = new Path("file:///img");
FileSystem fs = outdir.getFileSystem(conf);
if (fs.exists(outdir) && !fs.getFileStatus(outdir).isDirectory()) {
  throw new IllegalStateException(outdir + " exists as a file; remove it before fs2img");
}
if (!fs.exists(outdir) && !fs.mkdirs(outdir)) {
  throw new IllegalStateException("cannot create " + outdir + "; check parent permissions");
}

Try / catch

try {
  new ImageWriter(opts);
} catch (IOException e) {
  if (e.getMessage().startsWith("Failed to create output dir")) {
    // fix the -o path: conflicting file or unwritable parent, then retry
  }
}

Prevention

Pre-check the -o path: absent or an existing directory, and parent writable.
Use explicit schemes (file:/// or hdfs://) in -o to avoid surprise resolution.

When it happens

Trigger: Passing -o with a path occupied by an existing file; running without write permission on the output parent directory; a scheme in the output URI that maps to a read-only or unavailable FileSystem.

Common situations: A previous run left a file where the directory should be; output directed to a protected location (e.g. under /var without root); copy-pasted -o value with a typo'd scheme.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9e90adc7b08d2d3b. Report an issue: GitHub.