apache/hadoop · error · IOException

Unable to delete ${file}

Error message

Unable to delete ${file}

What it means

deleteFile removes a token or key file; a FileNotFoundException is treated as success (idempotent delete), but fs.delete returning false throws IOException 'Unable to delete'. This happens when the filesystem declines the delete without throwing: permission denial, snapshot/quota constraints, or the file being recreated concurrently by another writer.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryServerFileSystemStateStoreService.java:295

    byte[] data = new byte[(int)numBytes];
    FSDataInputStream in = fs.open(file);
    try {
      in.readFully(data);
    } finally {
      IOUtils.cleanupWithLogger(LOG, in);
    }
    return data;
  }

  private void deleteFile(Path file) throws IOException {
    boolean deleted;
    try {
      deleted = fs.delete(file, false);
    } catch (FileNotFoundException e) {
      deleted = true;
    }
    if (!deleted) {
      throw new IOException("Unable to delete " + file);
    }
  }

  private byte[] buildTokenData(MRDelegationTokenIdentifier tokenId,
      Long renewDate) throws IOException {
    ByteArrayOutputStream memStream = new ByteArrayOutputStream();
    DataOutputStream dataStream = new DataOutputStream(memStream);
    try {
      tokenId.write(dataStream);
      dataStream.writeLong(renewDate);
      dataStream.close();
      dataStream = null;
    } finally {
      IOUtils.cleanupWithLogger(LOG, dataStream);
    }
    return memStream.toByteArray();
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Check and fix delete permissions on the whole store tree for the mapred history user.
2. Identify and stop any second writer to the same store location.
3. Verify no HDFS snapshot, quota, or trash policy holds the file, then retry after remediation.
4. Restart JHS once the filesystem permits the operation.

try {
  store.removeToken(tokenId);
} catch (IOException e) {
  if (e.getMessage().startsWith("Unable to delete")) {
    // delete refused without exception: check permissions/snapshots, then retry
  }
}

When it happens

Trigger: JHS user lacks delete permission on the file or its parent; HDFS snapshot or quota semantics blocking removal; a concurrent second writer recreating the file; trash configuration interfering.

Common situations: Permission hardening after initial deployment; a second JHS sharing the store; store directory on a restricted mount.

Related errors

• Could not rename ${tmp} to ${tokenPath}
• Unexpected file in store: ${dir}
• Could not rename ${tmp} to ${file}
• Failed to delete {pTask}
• ${tokenPath} already exists

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d294e00c4fc6ff50. Report an issue: GitHub.