{"record":{"id":"3e48ce7d93fe32ff","repo":"apache/iceberg","slug":"cannot-remove-orphan-files-with-an-interval-less-t","errorCode":null,"errorMessage":"Cannot remove orphan files with an interval less than 24 hours. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval.","messagePattern":"Cannot remove orphan files with an interval less than 24 hours\\. Executing this procedure with a short interval may corrupt the table if other operations are happening at the same time\\. If you are absolutely confident that no concurrent operations will be affected by removing orphan files with such a short interval, you can use the Action API to remove orphan files with an arbitrary interval\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java","lineNumber":223,"sourceCode":"  private InternalRow[] toOutputRows(DeleteOrphanFiles.Result result) {\n    Iterable<String> orphanFileLocations = result.orphanFileLocations();\n\n    int orphanFileLocationsCount = Iterables.size(orphanFileLocations);\n    InternalRow[] rows = new InternalRow[orphanFileLocationsCount];\n\n    int index = 0;\n    for (String fileLocation : orphanFileLocations) {\n      rows[index] = newInternalRow(UTF8String.fromString(fileLocation));\n      index++;\n    }\n\n    return rows;\n  }\n\n  private void validateInterval(long olderThanMillis) {\n    long intervalMillis = System.currentTimeMillis() - olderThanMillis;\n    if (intervalMillis < TimeUnit.DAYS.toMillis(1)) {\n      throw new IllegalArgumentException(\n          \"Cannot remove orphan files with an interval less than 24 hours. Executing this \"\n              + \"procedure with a short interval may corrupt the table if other operations are happening \"\n              + \"at the same time. If you are absolutely confident that no concurrent operations will be \"\n              + \"affected by removing orphan files with such a short interval, you can use the Action API \"\n              + \"to remove orphan files with an arbitrary interval.\");\n    }\n  }\n\n  @Override\n  public String description() {\n    return \"RemoveOrphanFilesProcedure\";\n  }\n\n  private PrefixMismatchMode asPrefixMismatchMode(ProcedureInput input, ProcedureParameter param) {\n    String modeAsString = input.asString(param, null);\n    return (modeAsString == null) ? null : PrefixMismatchMode.fromString(modeAsString);\n  }\n}","sourceCodeStart":205,"sourceCodeEnd":241,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/RemoveOrphanFilesProcedure.java#L205-L241","documentation":"remove_orphan_files refuses to delete files newer than 24 hours because concurrent operations may still be writing them, which can corrupt tables. validateInterval computes now - older_than and throws IllegalArgumentException when the interval is under one day, pointing users who truly need aggressive cleanup to the Action API.","triggerScenarios":"Calling `CALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '<recent>')` where the timestamp is less than 24 hours in the past (or in the future).","commonSituations":"Attempting immediate cleanup after a failed job; misunderstanding that older_than defaults to now - 3 days and passing minutes/hours; CI cleanup scripts with aggressive retention; timezone/clock mistakes shrinking the interval.","solutions":["Set older_than at least 24 hours in the past, e.g. older_than => TIMESTAMP '2026-09-09 00:00:00'.","Keep the procedure default (3 days), which is already safe.","If a shorter interval is truly required, use the DeleteOrphanFiles Action API directly with the chosen interval, accepting the risk.","Validate the computed interval and timezone before invoking to avoid accidental short windows."],"exampleFix":"// before\nCALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '2026-09-11 10:00:00')\n// after\nCALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '2026-09-08 10:00:00')","handlingStrategy":"validation","validationCode":"long olderThanMillis = java.sql.Timestamp.valueOf(olderThan).getTime();\nif (System.currentTimeMillis() - olderThanMillis < java.util.concurrent.TimeUnit.DAYS.toMillis(1)) {\n  throw new IllegalArgumentException(\"older_than must be at least 24 hours in the past\");\n}","typeGuard":null,"tryCatchPattern":"try { spark.sql(\"CALL cat.system.remove_orphan_files(table => 't', older_than => TIMESTAMP '...')\"); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith(\"Cannot remove orphan files with an interval less than 24 hours\")) { /* move older_than further back or use Action API */ } throw e; }","preventionTips":["Keep older_than at least 24 hours (ideally 3 days, the default) in the past.","Compute timestamps in UTC to avoid timezone-induced short intervals.","Use the DeleteOrphanFiles Action API when an arbitrary interval is genuinely required.","Never point orphan-file cleanup at very recent timestamps in active tables."],"tags":["spark","procedures","orphan-files","safety-guard","validation"],"backgroundTag":"invalid-argument-value","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}