{"record":{"id":"0833daf3f5a7ccdf","repo":"apache/iceberg","slug":"deleted-only-of-files-from-table-using-bu","errorCode":null,"errorMessage":"Deleted only {} of {} files from table {} using bulk deletes","messagePattern":"Deleted only (.+?) of (.+?) files from table (.+?) using bulk deletes","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DeleteFilesProcessor.java","lineNumber":111,"sourceCode":"  public void processWatermark(Watermark mark) {\n    deleteFiles();\n  }\n\n  @Override\n  public void prepareSnapshotPreBarrier(long checkpointId) {\n    deleteFiles();\n  }\n\n  private void deleteFiles() {\n    try {\n      io.deleteFiles(filesToDelete);\n      LOG.info(\n          \"Deleted {} files from table {} using bulk deletes\", filesToDelete.size(), tableName);\n      succeededCounter.inc(filesToDelete.size());\n      filesToDelete.clear();\n    } catch (BulkDeletionFailureException e) {\n      int deletedFilesCount = filesToDelete.size() - e.numberFailedObjects();\n      LOG.warn(\n          \"Deleted only {} of {} files from table {} using bulk deletes\",\n          deletedFilesCount,\n          filesToDelete.size(),\n          tableName,\n          e);\n      succeededCounter.inc(deletedFilesCount);\n      failedCounter.inc(e.numberFailedObjects());\n    }\n  }\n}\n","sourceCodeStart":93,"sourceCodeEnd":122,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DeleteFilesProcessor.java#L93-L122","documentation":"This is a WARN log (not a thrown exception) emitted when a bulk delete of data/delete files partially fails: a BulkDeletionFailureException reports how many objects the object store refused to delete. The processor counts the files that were actually deleted (total minus numberFailedObjects), increments the succeeded counter, and continues. It signals incomplete cleanup of orphan/obsolete files rather than a pipeline failure.","triggerScenarios":"deleteFiles() performs a bulk delete of a batch of files and the underlying FileIO bulk-delete call throws BulkDeletionFailureException with numberFailedObjects() > 0. Called from processElement, processWatermark, and prepareSnapshotPreBarrier while the maintenance task is expiring/cleaning files.","commonSituations":"S3 eventual-consistency or throttling (503 SlowDown) during large batch deletes; files already removed by a concurrent maintenance run or orphan cleanup; object-lock/retention or versioned buckets rejecting DELETE; transient credentials/permission issues on part of the batch.","solutions":["Rerun the maintenance task (e.g. DELETE ORPHAN FILES / expire) — deletion is idempotent and remaining files will be retried on the next run.","Check object store access: verify credentials, IAM delete permissions, bucket versioning/object-lock, and lifecycle rules that may conflict.","Inspect the BulkDeletionFailureException/underlying cause for per-object errors (throttling, NoSuchKey) and reduce batch size or add retry/backoff on the FileIO client.","Ensure only one maintenance job runs concurrently against the table to avoid double-deletion races."],"exampleFix":"// before: bulk delete fails wholesale handling\nio.deleteFilesWithBulk(locations);\n// after: fall back to per-file delete for failures\ntry {\n  io.deleteFilesWithBulk(locations);\n} catch (BulkDeletionFailureException e) {\n  locations.removeAll(e.failedObjects());\n  for (String loc : e.failedObjects()) {\n    try { io.deleteFile(loc); } catch (RuntimeException ignored) { /* retry next run */ }\n  }\n}","handlingStrategy":"retry","validationCode":"// pre-check a sample location is deletable and bucket supports delete\nString sample = locations.iterator().next();\ntry (FileIO io = table.io()) {\n  Preconditions.checkArgument(io instanceof SupportsBulkOperations || locations.size() == 1,\n      \"FileIO %s does not support bulk deletes\", io.getClass().getSimpleName());\n}","typeGuard":"boolean supportsBulkDelete(FileIO io) { return io instanceof SupportsBulkOperations; }","tryCatchPattern":"try {\n  io.deleteFilesWithBulk(locations);\n} catch (BulkDeletionFailureException e) {\n  // partial success; reconcile and retry failed subset idempotently\n  Set<String> remaining = new HashSet<>(e.failedObjects());\n  scheduleRetry(remaining);\n}","preventionTips":["Run maintenance tasks idempotently and rerun them; partial deletes self-heal on the next run.","Use an S3/HDFS client tuned for bulk deletes (adequate concurrency, retry/throttling backoff).","Avoid overlapping maintenance jobs on the same table.","Monitor the succeededCounter vs. total to alert on repeated partial deletions."],"tags":["flink","file-deletion","object-store","partial-failure","maintenance"],"backgroundTag":"http-error-response","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}