apache/iceberg · error · IllegalArgumentException

Cannot remove orphan files with an interval less than 24 hou

Error message

Cannot remove orphan files with an interval less than 24 hours. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval.

What it means

RemoveOrphanFilesProcedure refuses to delete files newer than 24 hours by default, because concurrent writers may still be committing data files whose presence check would race the deletion. If the effective interval between older_than and now is under one day, validateInterval throws IllegalArgumentException and suggests using the low-level Action API if the operator accepts the risk.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java:229

  private InternalRow[] toOutputRows(DeleteOrphanFiles.Result result) {
    Iterable<String> orphanFileLocations = result.orphanFileLocations();

    int orphanFileLocationsCount = Iterables.size(orphanFileLocations);
    InternalRow[] rows = new InternalRow[orphanFileLocationsCount];

    int index = 0;
    for (String fileLocation : orphanFileLocations) {
      rows[index] = newInternalRow(UTF8String.fromString(fileLocation));
      index++;
    }

    return rows;
  }

  private void validateInterval(long olderThanMillis) {
    long intervalMillis = System.currentTimeMillis() - olderThanMillis;
    if (intervalMillis < TimeUnit.DAYS.toMillis(1)) {
      throw new IllegalArgumentException(
          "Cannot remove orphan files with an interval less than 24 hours. Executing this "
              + "procedure with a short interval may corrupt the table if other operations are happening "
              + "at the same time. If you are absolutely confident that no concurrent operations will be "
              + "affected by removing orphan files with such a short interval, you can use the Action API "
              + "to remove orphan files with an arbitrary interval.");
    }
  }

  @Override
  public String name() {
    return NAME;
  }

  @Override
  public String description() {
    return "RemoveOrphanFilesProcedure";
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Set older_than to at least 24 hours in the past, e.g. date_sub(now(), 1).
2. If you truly need a shorter interval, use the RemoveOrphanFiles Spark action directly (SparkActions.get().deleteOrphanFiles(table)) which allows arbitrary intervals.
3. Ensure no concurrent writes are running before using short intervals via the Action API.
4. Schedule periodic cleanup with day-old thresholds instead of ad-hoc immediate cleanup.

Example fix

-- before
CALL iceberg.system.remove_orphan_files(table => 'db.t', older_than => current_timestamp());
-- after
CALL iceberg.system.remove_orphan_files(table => 'db.t', older_than => date_sub(current_timestamp(), 1));

if (Timestamp.valueOf(olderThan).getTime() > System.currentTimeMillis() - TimeUnit.DAYS.toMillis(1)) throw new IllegalArgumentException("older_than must be at least 24h in the past");

try { removeOrphanFiles(...); } catch (IllegalArgumentException e) { if (e.getMessage().contains("interval less than 24 hours")) { /* widen interval or use Action API */ } else throw e; }

When it happens

Trigger: CALL iceberg.system.remove_orphan_files(table => 'db.t', older_than => TIMESTAMP 'now-ish') where older_than is less than 24 hours in the past — e.g. passing CURRENT_TIMESTAMP or a timestamp from earlier today.

Common situations: Trying to clean up files from a failed run minutes ago; scripting the procedure with older_than => current_timestamp() (the default-like extreme); misunderstanding that older_than is an absolute timestamp, not a duration.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Cannot remove orphan files with an interval less than 24 hou
• Cannot remove orphan files with an interval less than 24 hou
• Unable to determine whether certain files are orphan. Metada
• Could not list sub directories, reached maximum depth:
• Cannot remove orphan files with an interval less than 24 hou

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1f94debbbf497085. Report an issue: GitHub.