{"record":{"id":"a5e34df8c65fbe84","repo":"apache/iceberg","slug":"cannot-remove-orphan-files-with-an-interval-less-t-a5e34d","errorCode":null,"errorMessage":"Cannot remove orphan files with an interval less than 24 hours. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval.","messagePattern":"Cannot remove orphan files with an interval less than 24 hours\\. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time\\. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java","lineNumber":229,"sourceCode":"  private InternalRow[] toOutputRows(DeleteOrphanFiles.Result result) {\n    Iterable<String> orphanFileLocations = result.orphanFileLocations();\n\n    int orphanFileLocationsCount = Iterables.size(orphanFileLocations);\n    InternalRow[] rows = new InternalRow[orphanFileLocationsCount];\n\n    int index = 0;\n    for (String fileLocation : orphanFileLocations) {\n      rows[index] = newInternalRow(UTF8String.fromString(fileLocation));\n      index++;\n    }\n\n    return rows;\n  }\n\n  private void validateInterval(long olderThanMillis) {\n    long intervalMillis = System.currentTimeMillis() - olderThanMillis;\n    if (intervalMillis < TimeUnit.DAYS.toMillis(1)) {\n      throw new IllegalArgumentException(\n          \"Cannot remove orphan files with an interval less than 24 hours. Executing this \"\n              + \"procedure with a short interval may corrupt the table if other operations are happening \"\n              + \"at the same time. If you are absolutely confident that no concurrent operations will be \"\n              + \"affected by removing orphan files with such a short interval, you can use the Action API \"\n              + \"to remove orphan files with an arbitrary interval.\");\n    }\n  }\n\n  @Override\n  public String name() {\n    return NAME;\n  }\n\n  @Override\n  public String description() {\n    return \"RemoveOrphanFilesProcedure\";\n  }\n","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java#L211-L247","documentation":"RemoveOrphanFilesProcedure refuses to delete files newer than 24 hours. validateInterval computes now - older_than and throws IllegalArgumentException when it is under one day, because concurrent operations may still be writing files that would be deleted, corrupting the table.","triggerScenarios":"CALL ... remove_orphan_files with older_than => TIMESTAMP/INTERVAL making the cutoff less than 24 hours in the past (e.g. older_than => current_timestamp() - INTERVAL 1 HOUR).","commonSituations":"Trying to reclaim disk space quickly after a test; misunderstanding that older_than is an absolute timestamp (an ambiguous timezone can also shrink the effective interval); automated cleanup scripts with aggressive intervals.","solutions":["Use an interval of at least 24 hours: older_than => current_timestamp() - INTERVAL 1 DAYS","If you truly must use a shorter interval, call the Action API directly (SparkActions.get().deleteOrphanFiles(table).olderThan(millis)) after confirming no concurrent writes","Dry-run first with dry_run => true to see what would be deleted","Ensure clocks/timezones are consistent so the intended cutoff is actually >= 24h"],"exampleFix":"// before\nCALL cat.system.remove_orphan_files(table => 'cat.db.t', older_than => current_timestamp() - INTERVAL 1 HOURS);\n// after\nCALL cat.system.remove_orphan_files(table => 'cat.db.t', older_than => current_timestamp() - INTERVAL 1 DAYS, dry_run => true);","handlingStrategy":"validation","validationCode":"if (System.currentTimeMillis() - olderThanMillis < TimeUnit.DAYS.toMillis(1)) throw new IllegalArgumentException(\"older_than must be at least 24 hours in the past\")","typeGuard":null,"tryCatchPattern":"try { spark.sql(call) } catch { case e: IllegalArgumentException if e.getMessage.contains(\"interval less than 24 hours\") => /* widen interval or use Action API */ }","preventionTips":["Default to older_than >= now - 1 DAY","Run with dry_run => true first","Use the Action API (deleteOrphanFiles) only when you can guarantee no concurrent writes"],"tags":["spark","procedures","orphan-files","safety-guard"],"backgroundTag":"argument-out-of-range","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}