apache/iceberg · error · IllegalArgumentException

Cannot remove orphan files with an interval less than 24 hou

Error message

Cannot remove orphan files with an interval less than 24 hours. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval.

What it means

RemoveOrphanFilesProcedure guards against deleting files that concurrent writes may still consider live. It rejects older_than values that correspond to an interval shorter than 24 hours from now, because short windows can delete files belonging to in-flight commits, corrupting the table.

Source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java:229

  private InternalRow[] toOutputRows(DeleteOrphanFiles.Result result) {
    Iterable<String> orphanFileLocations = result.orphanFileLocations();

    int orphanFileLocationsCount = Iterables.size(orphanFileLocations);
    InternalRow[] rows = new InternalRow[orphanFileLocationsCount];

    int index = 0;
    for (String fileLocation : orphanFileLocations) {
      rows[index] = newInternalRow(UTF8String.fromString(fileLocation));
      index++;
    }

    return rows;
  }

  private void validateInterval(long olderThanMillis) {
    long intervalMillis = System.currentTimeMillis() - olderThanMillis;
    if (intervalMillis < TimeUnit.DAYS.toMillis(1)) {
      throw new IllegalArgumentException(
          "Cannot remove orphan files with an interval less than 24 hours. Executing this "
              + "procedure with a short interval may corrupt the table if other operations are happening "
              + "at the same time. If you are absolutely confident that no concurrent operations will be "
              + "affected by removing orphan files with such a short interval, you can use the Action API "
              + "to remove orphan files with an arbitrary interval.");
    }
  }

  @Override
  public String name() {
    return NAME;
  }

  @Override
  public String description() {
    return "RemoveOrphanFilesProcedure";
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Set older_than to a timestamp at least 24 hours in the past (e.g. days-ago or older).
2. If a shorter interval is truly needed, use the Action API (RemoveOrphanFilesSparkAction) directly, which allows arbitrary intervals after you confirm no concurrent operations.
3. Reduce cleanup frequency rather than the interval, and run cleanup during write-quiet periods.

Example fix

// before
call remove_orphan_files(table => 'db.t', older_than => current_timestamp())
// after
call remove_orphan_files(table => 'db.t', older_than => date_sub(current_timestamp(), 3))

val olderThan = date_sub(current_timestamp(), 3)
require(timestampdiff(SECOND, olderThan, current_timestamp()) >= 86400, "interval must be >= 24h")

try {
  spark.sql(s"CALL cat.system.remove_orphan_files(table => '$t', older_than => $ts)")
} catch {
  case e: IllegalArgumentException if e.getMessage.contains("interval less than 24 hours") =>
    logger.error("Use the Action API if a shorter interval is truly safe")
}

When it happens

Trigger: Calling call remove_orphan_files(table => ..., older_than => TIMESTAMP '<less than 24h ago>') — e.g. older_than set to current_timestamp() or a few hours back.

Common situations: Trying to reclaim disk space quickly in an emergency; automated cleanup jobs with too-aggressive retention; misunderstanding that older_than must be well in the past.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Cannot remove orphan files with an interval less than 24 hou
• Cannot remove orphan files with an interval less than 24 hou
• Cannot remove orphan files with an interval less than 24 hou
• this.getClass().getName() + " does not implement rewritePosi
• this.getClass().getName() + " does not implement computeTabl

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/84503b0135c5c09b. Report an issue: GitHub.