{"record":{"id":"1f94debbbf497085","repo":"apache/iceberg","slug":"cannot-remove-orphan-files-with-an-interval-less-t-1f94de","errorCode":null,"errorMessage":"Cannot remove orphan files with an interval less than 24 hours. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval.","messagePattern":"Cannot remove orphan files with an interval less than 24 hours\\. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time\\. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java","lineNumber":229,"sourceCode":"  private InternalRow[] toOutputRows(DeleteOrphanFiles.Result result) {\n    Iterable<String> orphanFileLocations = result.orphanFileLocations();\n\n    int orphanFileLocationsCount = Iterables.size(orphanFileLocations);\n    InternalRow[] rows = new InternalRow[orphanFileLocationsCount];\n\n    int index = 0;\n    for (String fileLocation : orphanFileLocations) {\n      rows[index] = newInternalRow(UTF8String.fromString(fileLocation));\n      index++;\n    }\n\n    return rows;\n  }\n\n  private void validateInterval(long olderThanMillis) {\n    long intervalMillis = System.currentTimeMillis() - olderThanMillis;\n    if (intervalMillis < TimeUnit.DAYS.toMillis(1)) {\n      throw new IllegalArgumentException(\n          \"Cannot remove orphan files with an interval less than 24 hours. Executing this \"\n              + \"procedure with a short interval may corrupt the table if other operations are happening \"\n              + \"at the same time. If you are absolutely confident that no concurrent operations will be \"\n              + \"affected by removing orphan files with such a short interval, you can use the Action API \"\n              + \"to remove orphan files with an arbitrary interval.\");\n    }\n  }\n\n  @Override\n  public String name() {\n    return NAME;\n  }\n\n  @Override\n  public String description() {\n    return \"RemoveOrphanFilesProcedure\";\n  }\n","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java#L211-L247","documentation":"RemoveOrphanFilesProcedure refuses to delete files newer than 24 hours by default, because concurrent writers may still be committing data files whose presence check would race the deletion. If the effective interval between older_than and now is under one day, validateInterval throws IllegalArgumentException and suggests using the low-level Action API if the operator accepts the risk.","triggerScenarios":"CALL iceberg.system.remove_orphan_files(table => 'db.t', older_than => TIMESTAMP 'now-ish') where older_than is less than 24 hours in the past — e.g. passing CURRENT_TIMESTAMP or a timestamp from earlier today.","commonSituations":"Trying to clean up files from a failed run minutes ago; scripting the procedure with older_than => current_timestamp() (the default-like extreme); misunderstanding that older_than is an absolute timestamp, not a duration.","solutions":["Set older_than to at least 24 hours in the past, e.g. date_sub(now(), 1).","If you truly need a shorter interval, use the RemoveOrphanFiles Spark action directly (SparkActions.get().deleteOrphanFiles(table)) which allows arbitrary intervals.","Ensure no concurrent writes are running before using short intervals via the Action API.","Schedule periodic cleanup with day-old thresholds instead of ad-hoc immediate cleanup."],"exampleFix":"-- before\nCALL iceberg.system.remove_orphan_files(table => 'db.t', older_than => current_timestamp());\n-- after\nCALL iceberg.system.remove_orphan_files(table => 'db.t', older_than => date_sub(current_timestamp(), 1));","handlingStrategy":"validation","validationCode":"if (Timestamp.valueOf(olderThan).getTime() > System.currentTimeMillis() - TimeUnit.DAYS.toMillis(1)) throw new IllegalArgumentException(\"older_than must be at least 24h in the past\");","typeGuard":null,"tryCatchPattern":"try { removeOrphanFiles(...); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"interval less than 24 hours\")) { /* widen interval or use Action API */ } else throw e; }","preventionTips":["Default older_than to now() - 1 day or more.","Remember older_than is an absolute timestamp, not a duration.","Use the RemoveOrphanFiles Action API only when concurrency is ruled out."],"tags":["spark","maintenance","orphan-files"],"backgroundTag":"invalid-argument-value","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}