{"record":{"id":"d2980763cd60d2c4","repo":"apache/iceberg","slug":"failed-to-load-committed-snapshot-skipping-manife","errorCode":null,"errorMessage":"Failed to load committed snapshot, skipping manifest clean-up","messagePattern":"Failed to load committed snapshot, skipping manifest clean-up","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/SnapshotProducer.java","lineNumber":556,"sourceCode":"        // id in case another commit was added between this commit and the refresh.\n        // it might not be known which commit attempt succeeded in some cases, so this only cleans\n        // up the one that actually did succeed.\n        Snapshot saved = ops.refresh().snapshot(newSnapshotId.get());\n        if (saved != null) {\n          if (cleanupAfterCommit()) {\n            cleanUncommitted(Sets.newHashSet(saved.allManifests(ops.io())));\n          }\n\n          // also clean up unused manifest lists created by multiple attempts\n          for (String manifestList : manifestLists) {\n            if (!saved.manifestListLocation().equals(manifestList)) {\n              deleteFile(manifestList);\n            }\n          }\n        } else {\n          // saved may not be present if the latest metadata couldn't be loaded due to eventual\n          // consistency problems in refresh. in that case, don't clean up.\n          LOG.warn(\"Failed to load committed snapshot, skipping manifest clean-up\");\n        }\n      } catch (Throwable e) {\n        LOG.warn(\n            \"Failed to load committed table metadata or during cleanup, skipping further cleanup\",\n            e);\n      }\n    }\n\n    try {\n      notifyListeners();\n    } catch (Throwable e) {\n      LOG.warn(\"Failed to notify event listeners\", e);\n    }\n  }\n\n  private void notifyListeners() {\n    try {\n      Object event = updateEvent();","sourceCodeStart":538,"sourceCodeEnd":574,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/SnapshotProducer.java#L538-L574","documentation":"This warning is emitted during SnapshotProducer.commit()'s cleanup phase when re-reading the table's latest metadata fails to find the just-committed snapshot. It typically indicates eventual-consistency problems in the underlying catalog refresh, so the newly written manifest lists cannot be confirmed as committed. Because commit state cannot be verified, the producer deliberately skips manifest clean-up to avoid deleting files that may belong to the committed snapshot.","triggerScenarios":"table.refresh() or the catalog read inside commit cleanup returns metadata that does not contain the committed snapshot, usually because the catalog store exhibits read-after-write (eventual) consistency lag during a commit via SnapshotProducer.commit().","commonSituations":"Committing to tables in eventually-consistent object stores (e.g. S3 without strong consistency guarantees) or catalogs with replication lag; concurrent commits from other jobs racing with cleanup; network blips during metadata re-read.","solutions":["Verify the catalog/storage backend provides strong read-after-write consistency (or enable an Iceberg lock/catalog implementation that does).","Check for concurrent writers fighting over the same table and ensure a proper catalog locking mechanism is in place.","Retry the commit; the warning is non-fatal — orphan manifests can be removed later with ExpireSnapshots or removeOrphanFiles.","Inspect catalog logs to confirm whether the commit actually landed; if it did, no data loss occurred, only leftover files."],"exampleFix":"// before\nTable table = catalog.loadTable(identifier);\ntable.newAppend().appendFile(dataFile).commit(); // catalog refresh is eventually consistent\n// after\nTable table = catalog.loadTable(identifier);\ntable.refresh(); // ensure fresh metadata before committing\ntable.newAppend().appendFile(dataFile).commit();\n// then run removeOrphanFiles/expireSnapshots periodically to clean leftovers","handlingStrategy":"validation","validationCode":"if (!table.operations().current().snapshot(table.currentSnapshot().snapshotId()).isPresent()) {\n  throw new IllegalStateException(\"Catalog refresh did not return the committed snapshot; skipping dependent cleanup\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use a catalog with strong read-after-write consistency","Periodically run RemoveOrphanFiles/ExpireSnapshots to clean leftover manifests","Avoid many concurrent writers on the same table without proper locking"],"tags":["commit","eventual-consistency","cleanup","catalog"],"backgroundTag":"upstream-api-error","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}