What causes "Failed to delete file: %s" in apache/iceberg?

Inspect the wrapped cause via getCause() for the concrete FileSystem error and address it (credentials, permissions, connectivity).

apache/iceberg · error · RuntimeIOException

Failed to delete file: %s

Error message

Failed to delete file: %s

What it means

HadoopFileIO.deleteFile deletes a single (non-directory) file at the given path using FileSystem.delete(path, false). Any IOException from the underlying filesystem — permission problems, missing credentials, network errors — is wrapped in this RuntimeIOException naming the path.

Source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java:103

  @Override
  public InputFile newInputFile(String path, long length) {
    return HadoopInputFile.fromLocation(path, length, getConf());
  }

  @Override
  public OutputFile newOutputFile(String path) {
    return HadoopOutputFile.fromPath(new Path(path), getConf());
  }

  @Override
  public void deleteFile(String path) {
    Path toDelete = new Path(path);
    FileSystem fs = Util.getFs(toDelete, getConf());
    try {
      fs.delete(toDelete, false /* not recursive */);
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to delete file: %s", path);
    }
  }

  @Override
  public Map<String, String> properties() {
    return properties.immutableMap();
  }

  @Override
  public void setConf(Configuration conf) {
    this.hadoopConf = new SerializableConfiguration(conf);
  }

  @Override
  public Configuration getConf() {
    // Create a default hadoopConf as it is required for the object to be valid.
    // E.g. newInputFile would throw NPE with getConf() otherwise.
    if (hadoopConf == null) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect the wrapped cause via getCause() for the concrete FileSystem error and address it (credentials, permissions, connectivity).
Verify the file still exists before deleting if concurrent deletes are possible.
Confirm the configured Hadoop conf for HadoopFileIO has correct fs.<scheme>.impl and auth settings.
Retry after transient network/HDFS issues; use idempotent cleanup that tolerates already-deleted files.

Example fix

// before
io.deleteFile("hdfs://nn/warehouse/db/table/data-0001.parquet");
// after
try {
  io.deleteFile(path);
} catch (RuntimeIOException e) {
  if (io.newInputFile(path).exists()) throw e; // genuinely failed
  LOG.info("File already deleted: {}", path);
}

Defensive patterns

Strategy: try-catch

Validate before calling

InputFile check = io.newInputFile(path);
boolean exists = check.exists(); // cheap pre-check before delete

Try / catch

try {
  io.deleteFile(path);
} catch (RuntimeIOException e) {
  LOG.error("Delete failed for {}: cause={}", path, e.getCause());
  // retry on transient causes; skip if file already absent
}

Prevention

Pre-check file existence when concurrent deleters exist.
Keep cloud credentials valid for the whole job duration.
Run orphan-file cleanup exclusively, not concurrently with writers.

When it happens

Trigger: Calling deleteFile(path) when the filesystem throws IOException: no permission to delete, HDFS unavailable, S3/GCS auth failure, or the file was already removed concurrently (some FS implementations report this as IOException rather than returning false).

Common situations: Expired cloud credentials mid-job; deleteOrphanFiles cleanup racing with another writer; HDFS in safe mode; region/bucket misconfiguration so the FS client cannot reach the storage backend.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/2e88848cd29f2086. Report an issue: GitHub.