{"record":{"id":"2f439fca9a2dfcdb","repo":"apache/beam","slug":"failed-to-collect-file-statuses-for-snapshot","errorCode":null,"errorMessage":"Failed to collect file statuses for snapshot {}","messagePattern":"Failed to collect file statuses for snapshot (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/iceberg/BeamBaseIncrementalChangelogScan.java","lineNumber":443,"sourceCode":"              if (!changedDataManifests.isEmpty()) {\n                ManifestGroup changedGroup =\n                    new ManifestGroup(table().io(), changedDataManifests, ImmutableList.of())\n                        .specsById(table().specs())\n                        .caseSensitive(isCaseSensitive())\n                        .select(scanColumns())\n                        .filterData(filter())\n                        .ignoreExisting()\n                        .columnsToKeepStats(columnsToKeepStats());\n\n                try (CloseableIterable<ManifestEntry<DataFile>> entries = changedGroup.entries()) {\n                  for (ManifestEntry<DataFile> entry : entries) {\n                    if (changelogSnapshotIds.contains(entry.snapshotId())) {\n                      fileStatuses.put(entry.file().location(), entry.status());\n                      localAffected.add(entry.file().specId(), entry.file().partition());\n                    }\n                  }\n                } catch (Exception e) {\n                  throw new RuntimeException(\n                      \"Failed to collect file statuses for snapshot \" + snapshot.snapshotId(), e);\n                }\n              }\n\n              fileStatusBySnapshot.put(snapshot.snapshotId(), fileStatuses);\n              localPartitionsQueue.add(localAffected);\n            });\n\n    PartitionSet globalAffected = PartitionSet.create(table().specs());\n    for (PartitionSet local : localPartitionsQueue) {\n      globalAffected.addAll(local);\n    }\n\n    return Pair.of(fileStatusBySnapshot, globalAffected);\n  }\n\n  private List<ManifestFile> pruneManifestsByAffectedPartitions(\n      List<ManifestFile> manifests, PartitionSet affectedPartitions) {","sourceCodeStart":425,"sourceCodeEnd":461,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/iceberg/BeamBaseIncrementalChangelogScan.java#L425-L461","documentation":"buildFileStatusBySnapshot iterates a snapshot's manifest entries to map file locations to entry statuses for changelog snapshots. Any exception while reading manifest content (IO error, corrupt manifest, deserialization failure) is rethrown as a RuntimeException naming the snapshot id.","triggerScenarios":"Reading manifest entries of snapshot snapshotId() during an incremental changelog scan when the manifest file is unreadable, deleted, or its data/sequence metadata cannot be deserialized.","commonSituations":"Underlying files removed by expireSnapshots/rewriteManifests while the scan runs; GCS/S3/HDFS transient I/O failures; corrupt manifests from a failed commit.","solutions":["Check the wrapped cause for the underlying IO/deserialization error and verify the manifest file exists and is readable on the filesystem","Retry the scan; transient object-store failures (503, throttling) commonly cause this","Verify no concurrent expireSnapshots/rewriteJobs deleted the manifests referenced by the scanned snapshots; pause retention cleanup or rerun against a valid snapshot range"],"exampleFix":"// before\nscan = table.newIncrementalChangelogScan().fromSnapshotId(oldSnap).toSnapshotId(headSnap); // oldSnap manifests expired\n// after\nlong validFrom = table.currentSnapshot().getParentSnapshotId();\nscan = table.newIncrementalChangelogScan().fromSnapshotId(validFrom).toSnapshotId(table.currentSnapshot().snapshotId());","handlingStrategy":"retry","validationCode":"// before scan: confirm all snapshots in range still exist\nfor (Snapshot s : snapshotsInRange) {\n  checkState(table.snapshot(s.snapshotId()) != null, \"snapshot expired: \" + s.snapshotId());\n}","typeGuard":null,"tryCatchPattern":"try {\n  table.newIncrementalChangelogScan()...planFiles();\n} catch (RuntimeException e) {\n  if (e.getMessage().startsWith(\"Failed to collect file statuses\")) {\n    // retry with backoff; if persistent, widen retention and re-run\n  } else throw e;\n}","preventionTips":["Keep min-snapshot-retention longer than the changelog scan window","Avoid running expireSnapshots concurrently with incremental scans","Configure storage-client retries for the object store backing the table"],"tags":["iceberg","manifest-read","io"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}