apache/hadoop · error · PathIOException

Input/output error

Error message

Input/output error

What it means

PathIOException with the POSIX EIO text 'Input/output error' (PathIOException.java default message, rendered as `path`: Input/output error) thrown by Rm.processPath (Delete.java:124) when FileSystem.delete(path, recursive) returns false instead of throwing — the filesystem acknowledged the call but did not remove the path. This is a catch-all for delete failures the FS implementation signals via boolean rather than an exception.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Delete.java:124

    protected void processNonexistentPath(PathData item) throws IOException {
      if (!ignoreFNF) super.processNonexistentPath(item);
    }

    @Override
    protected void processPath(PathData item) throws IOException {
      if (item.stat.isDirectory() && !deleteDirs) {
        throw new PathIsDirectoryException(item.toString());
      }

      // TODO: if the user wants the trash to be used but there is any
      // problem (ie. creating the trash dir, moving the item to be deleted,
      // etc), then the path will just be deleted because moveToTrash returns
      // false and it falls thru to fs.delete.  this doesn't seem right
      if (moveToTrash(item) || !canBeSafelyDeleted(item)) {
        return;
      }
      if (!item.fs.delete(item.path, deleteDirs)) {
        throw new PathIOException(item.toString());
      }
      out.println("Deleted " + item);
    }

    private boolean canBeSafelyDeleted(PathData item)
        throws IOException {
      boolean shouldDelete = true;
      if (safeDelete) {
        final long deleteLimit = getConf().getLong(
            HADOOP_SHELL_SAFELY_DELETE_LIMIT_NUM_FILES,
            HADOOP_SHELL_SAFELY_DELETE_LIMIT_NUM_FILES_DEFAULT);
        if (deleteLimit > 0) {
          ContentSummary cs = item.fs.getContentSummary(item.path);
          final long numFiles = cs.getFileCount();
          if (numFiles > deleteLimit) {
            if (!ToolRunner.confirmPrompt("Proceed deleting " + numFiles +
                " files?")) {
              System.err.println("Delete aborted at user request.\n");

View on GitHub (pinned to 2add963021)

Solutions

1. Verify permissions: 'hdfs dfs -ls /parent' and check the owner/group bits; fix with hdfs dfs -chmod/-chown as needed
2. Check whether the path still exists after the failure ('hdfs dfs -ls path') — if it is gone, the delete raced and nothing is wrong
3. Retry once after permissions are fixed; inspect NameNode logs if it persists
4. If a concurrent job recreates the path, coordinate deletion (stop the writer first) instead of retrying

Example fix

// before
if (!fs.delete(path, false)) { /* ignored */ }

// after
try {
  if (!fs.delete(path, false)) {
    throw new PathIOException(path.toString());
  }
} catch (PathIOException e) {
  LOG.warn("delete failed for {}: {}", path, e.getMessage());
  // re-check existence + permissions, then retry once
}

if (fs.exists(path)) {
  FileStatus parent = fs.getFileStatus(path.getParent());
  // require write access on parent before attempting delete
}

try {
  if (!fs.delete(path, recursive)) {
    throw new PathIOException(path.toString());
  }
} catch (PathIOException e) {
  if (!fs.exists(path)) return;      // raced: already gone
  // verify/repair parent permissions, then retry once
  if (!fs.delete(path, recursive)) throw e;
}

When it happens

Trigger: fs.delete returning false: missing write permission on the file or its parent directory (HDFS permissions), the path being re-created by a concurrent writer between check and delete, viewfs/mount-table oddities, or a filesystem driver bug. Note this line is only reached after moveToTrash and the safe-delete gauge pass, so trash is not the culprit here.

Common situations: Cleanup jobs running as a user without write rights on the parent HDFS directory; racing MapReduce/Spark tasks that recreate output files; intermittent cluster issues where the NameNode answers listStatus but delete fails.

Related errors

• ${msg}. Consider using -skipTrash option
• Error while running command to get file permissions : " + St
• Is a directory
• Failed to delete {pTask}
• Directory {} is not empty.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b62ad375922395c4. Report an issue: GitHub.