What causes "Failed to delete file: %s" in apache/iceberg?

Inspect the IOException cause; distinguish permission vs connectivity vs throttling errors.

apache/iceberg · error · RuntimeIOException

Failed to delete file: %s

Error message

Failed to delete file: %s

What it means

HadoopCatalog.dropTable wraps IOExceptions from the recursive table-directory delete into a RuntimeIOException. This happens after the table metadata was confirmed present (and possibly purged via dropTableData), so a failure can leave a partially deleted table: data files may be gone while the metadata directory remains.

Source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java:270

    }

    Path tablePath = new Path(defaultWarehouseLocation(identifier));
    TableOperations ops = newTableOps(identifier);
    TableMetadata lastMetadata = ops.current();
    try {
      if (lastMetadata == null) {
        LOG.debug("Not an iceberg table: {}", identifier);
        return false;
      } else {
        if (purge) {
          // Since the data files and the metadata files may store in different locations,
          // so it has to call dropTableData to force delete the data file.
          CatalogUtil.dropTableData(ops.io(), lastMetadata);
        }
        return fs.delete(tablePath, true /* recursive */);
      }
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to delete file: %s", tablePath);
    }
  }

  @Override
  public void renameTable(TableIdentifier from, TableIdentifier to) {
    throw new UnsupportedOperationException("Cannot rename Hadoop tables");
  }

  @Override
  public void createNamespace(Namespace namespace, Map<String, String> meta) {
    Preconditions.checkArgument(
        !namespace.isEmpty(), "Cannot create namespace with invalid name: %s", namespace);
    if (!meta.isEmpty()) {
      throw new UnsupportedOperationException(
          "Cannot create namespace " + namespace + ": metadata is not supported");
    }

    Path nsPath = new Path(warehouseLocation, SLASH.join(namespace.levels()));

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect the IOException cause; distinguish permission vs connectivity vs throttling errors.
Retry the dropTable call — it is safe to re-run: a second call finds metadata missing and returns false or completes cleanup.
For cloud stores, apply exponential backoff / retry policies and check for rate-limit errors.
Verify the caller has delete permission on the table directory and its parent.
If data files were purged but metadata remains, re-run dropTable with purge=false to remove leftover metadata.

Example fix

// before: single attempt
boolean dropped = catalog.dropTable(ident, true);

// after: retry transient failures
try {
  catalog.dropTable(ident, true);
} catch (RuntimeIOException e) {
  Tasks.foreach(ident).retry(3).suppressExceptions().run(i -> catalog.dropTable(i, true));
}

Defensive patterns

Strategy: retry

Validate before calling

// No pre-check possible for I/O failure; ensure table exists first:
if (!catalog.tableExists(ident)) { return; }

Try / catch

try { catalog.dropTable(ident, true); } catch (RuntimeIOException e) { Tasks.foreach(ident).retry(3).exponentialBackoff(100, 10000).run(i -> catalog.dropTable(i, true)); }

Prevention

Ensure delete permissions on table and parent directories
Use retry policies for cloud-store deletes
Re-run dropTable after partial failure — it is idempotent

When it happens

Trigger: Calling catalog.dropTable(identifier, purge) when fs.delete(tablePath, true) throws IOException: filesystem unavailable, HDFS lease/permission issues, cloud-store throttling, or errors while dropTableData deletes data files in a different location.

Common situations: HDFS NameNode downtime mid-delete, S3 rate limiting (503 Slow Down) during bulk data-file deletion, permission revoked on the table directory, or object-store eventual consistency conflicts.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/9a34bd50b25e4c59. Report an issue: GitHub.