apache/iceberg · error · IllegalArgumentException

Cannot remove orphan files with an interval less than 24 hou

Error message

Cannot remove orphan files with an interval less than 24 hours. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval.

What it means

remove_orphan_files refuses to delete files newer than 24 hours because concurrent operations may still be writing them, which can corrupt tables. validateInterval computes now - older_than and throws IllegalArgumentException when the interval is under one day, pointing users who truly need aggressive cleanup to the Action API.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java:223

  private InternalRow[] toOutputRows(DeleteOrphanFiles.Result result) {
    Iterable<String> orphanFileLocations = result.orphanFileLocations();

    int orphanFileLocationsCount = Iterables.size(orphanFileLocations);
    InternalRow[] rows = new InternalRow[orphanFileLocationsCount];

    int index = 0;
    for (String fileLocation : orphanFileLocations) {
      rows[index] = newInternalRow(UTF8String.fromString(fileLocation));
      index++;
    }

    return rows;
  }

  private void validateInterval(long olderThanMillis) {
    long intervalMillis = System.currentTimeMillis() - olderThanMillis;
    if (intervalMillis < TimeUnit.DAYS.toMillis(1)) {
      throw new IllegalArgumentException(
          "Cannot remove orphan files with an interval less than 24 hours. Executing this "
              + "procedure with a short interval may corrupt the table if other operations are happening "
              + "at the same time. If you are absolutely confident that no concurrent operations will be "
              + "affected by removing orphan files with such a short interval, you can use the Action API "
              + "to remove orphan files with an arbitrary interval.");
    }
  }

  @Override
  public String description() {
    return "RemoveOrphanFilesProcedure";
  }

  private PrefixMismatchMode asPrefixMismatchMode(ProcedureInput input, ProcedureParameter param) {
    String modeAsString = input.asString(param, null);
    return (modeAsString == null) ? null : PrefixMismatchMode.fromString(modeAsString);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

Set older_than at least 24 hours in the past, e.g. older_than => TIMESTAMP '2026-09-09 00:00:00'.
Keep the procedure default (3 days), which is already safe.
If a shorter interval is truly required, use the DeleteOrphanFiles Action API directly with the chosen interval, accepting the risk.
Validate the computed interval and timezone before invoking to avoid accidental short windows.

Example fix

// before
CALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '2026-09-11 10:00:00')
// after
CALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '2026-09-08 10:00:00')

Defensive patterns

Strategy: validation

Validate before calling

long olderThanMillis = java.sql.Timestamp.valueOf(olderThan).getTime();
if (System.currentTimeMillis() - olderThanMillis < java.util.concurrent.TimeUnit.DAYS.toMillis(1)) {
  throw new IllegalArgumentException("older_than must be at least 24 hours in the past");
}

Try / catch

try { spark.sql("CALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '...')"); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Cannot remove orphan files with an interval less than 24 hours")) { /* move older_than further back or use Action API */ } throw e; }

Prevention

Keep older_than at least 24 hours (ideally 3 days, the default) in the past.
Compute timestamps in UTC to avoid timezone-induced short intervals.
Use the DeleteOrphanFiles Action API when an arbitrary interval is genuinely required.
Never point orphan-file cleanup at very recent timestamps in active tables.

When it happens

Trigger: Calling `CALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '<recent>')` where the timestamp is less than 24 hours in the past (or in the future).

Common situations: Attempting immediate cleanup after a failed job; misunderstanding that older_than defaults to now - 3 days and passing minutes/hours; CI cleanup scripts with aggressive retention; timezone/clock mistakes shrinking the interval.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3e48ce7d93fe32ff. Report an issue: GitHub.