{"record":{"id":"84503b0135c5c09b","repo":"apache/iceberg","slug":"cannot-remove-orphan-files-with-an-interval-less-t-84503b","errorCode":null,"errorMessage":"Cannot remove orphan files with an interval less than 24 hours. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval.","messagePattern":"Cannot remove orphan files with an interval less than 24 hours\\. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time\\. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval\\.","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java","lineNumber":229,"sourceCode":"  private InternalRow[] toOutputRows(DeleteOrphanFiles.Result result) {\n    Iterable<String> orphanFileLocations = result.orphanFileLocations();\n\n    int orphanFileLocationsCount = Iterables.size(orphanFileLocations);\n    InternalRow[] rows = new InternalRow[orphanFileLocationsCount];\n\n    int index = 0;\n    for (String fileLocation : orphanFileLocations) {\n      rows[index] = newInternalRow(UTF8String.fromString(fileLocation));\n      index++;\n    }\n\n    return rows;\n  }\n\n  private void validateInterval(long olderThanMillis) {\n    long intervalMillis = System.currentTimeMillis() - olderThanMillis;\n    if (intervalMillis < TimeUnit.DAYS.toMillis(1)) {\n      throw new IllegalArgumentException(\n          \"Cannot remove orphan files with an interval less than 24 hours. Executing this \"\n              + \"procedure with a short interval may corrupt the table if other operations are happening \"\n              + \"at the same time. If you are absolutely confident that no concurrent operations will be \"\n              + \"affected by removing orphan files with such a short interval, you can use the Action API \"\n              + \"to remove orphan files with an arbitrary interval.\");\n    }\n  }\n\n  @Override\n  public String name() {\n    return NAME;\n  }\n\n  @Override\n  public String description() {\n    return \"RemoveOrphanFilesProcedure\";\n  }\n","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java#L211-L247","documentation":"RemoveOrphanFilesProcedure guards against deleting files that concurrent writes may still consider live. It rejects older_than values that correspond to an interval shorter than 24 hours from now, because short windows can delete files belonging to in-flight commits, corrupting the table.","triggerScenarios":"Calling call remove_orphan_files(table => ..., older_than => TIMESTAMP '<less than 24h ago>') — e.g. older_than set to current_timestamp() or a few hours back.","commonSituations":"Trying to reclaim disk space quickly in an emergency; automated cleanup jobs with too-aggressive retention; misunderstanding that older_than must be well in the past.","solutions":["Set older_than to a timestamp at least 24 hours in the past (e.g. days-ago or older).","If a shorter interval is truly needed, use the Action API (RemoveOrphanFilesSparkAction) directly, which allows arbitrary intervals after you confirm no concurrent operations.","Reduce cleanup frequency rather than the interval, and run cleanup during write-quiet periods."],"exampleFix":"// before\ncall remove_orphan_files(table => 'db.t', older_than => current_timestamp())\n// after\ncall remove_orphan_files(table => 'db.t', older_than => date_sub(current_timestamp(), 3))","handlingStrategy":"validation","validationCode":"val olderThan = date_sub(current_timestamp(), 3)\nrequire(timestampdiff(SECOND, olderThan, current_timestamp()) >= 86400, \"interval must be >= 24h\")","typeGuard":null,"tryCatchPattern":"try {\n  spark.sql(s\"CALL cat.system.remove_orphan_files(table => '$t', older_than => $ts)\")\n} catch {\n  case e: IllegalArgumentException if e.getMessage.contains(\"interval less than 24 hours\") =>\n    logger.error(\"Use the Action API if a shorter interval is truly safe\")\n}","preventionTips":["Always pass older_than at least 24h in the past","Prefer 3-7 day retention for orphan file cleanup","Schedule cleanup during write-quiet windows"],"tags":["spark","maintenance","safety-guard"],"backgroundTag":"invalid-config-value","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}