{"record":{"id":"a88300ce60c75a6c","repo":"apache/beam","slug":"failed-to-read-delete-manifest-manifest-path","errorCode":null,"errorMessage":"Failed to read delete manifest: ${manifest.path()}","messagePattern":"Failed to read delete manifest: (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/iceberg/BeamBaseIncrementalChangelogScan.java","lineNumber":761,"sourceCode":"        ManifestFiles.readDeleteManifest(manifest, table().io(), table().specs())) {\n      for (ManifestEntry<DeleteFile> entry : reader.entries()) {\n        if (entry.status() == ManifestEntry.Status.DELETED\n            && entry.snapshotId().equals(targetSnapshotId)) {\n          DeleteFile file = entry.file();\n\n          if (!partitionMatchesFilter(file)) {\n            continue;\n          }\n\n          Set<Integer> columns =\n              file.content() == FileContent.POSITION_DELETES\n                  ? Set.of(MetadataColumns.DELETE_FILE_PATH.fieldId())\n                  : Set.copyOf(file.equalityFieldIds());\n          deleteFiles.add(ContentFileUtil.copy(file, true, columns));\n        }\n      }\n    } catch (Exception e) {\n      throw new RuntimeException(\"Failed to read delete manifest: \" + manifest.path(), e);\n    }\n\n    return deleteFiles;\n  }\n\n  /**\n   * Prunes delete manifests based on partition filter to avoid processing irrelevant manifests.\n   * This significantly improves performance when only a subset of partitions are relevant to the\n   * scan.\n   *\n   * @param manifests all delete manifests to consider\n   * @return list of manifests that might contain relevant delete files\n   */\n  private List<ManifestFile> pruneManifestsByPartition(List<ManifestFile> manifests) {\n    Expression currentFilter = filter();\n\n    // If there's no filter, return all manifests\n    if (currentFilter == null || currentFilter.equals(Expressions.alwaysTrue())) {","sourceCodeStart":743,"sourceCodeEnd":779,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/iceberg/BeamBaseIncrementalChangelogScan.java#L743-L779","documentation":"This method reads a delete manifest and copies applicable delete files (with reduced columns for the relevant equality/delete path fields). Any exception while iterating manifest entries is wrapped in a RuntimeException naming the manifest path.","triggerScenarios":"Reading a manifest of type DELETES during changelog task creation when the manifest is unreadable, was deleted by retention, or its entries fail to deserialize.","commonSituations":"Manifest files removed by expireSnapshots mid-scan; corrupted manifests; transient GCS/S3 errors; permission changes on the metadata directory.","solutions":["Open the manifest at the reported path directly to confirm it exists and is readable; inspect the wrapped cause for the real error","Retry the scan to rule out transient object-store failures","Ensure snapshot/manifest retention policies keep all manifests in the scanned range for the duration of the pipeline"],"exampleFix":"// before\ntable.expireSnapshots().expireOlderThan(System.currentTimeMillis()).commit(); // deletes manifests scan still needs\n// after\nlong watermark = System.currentTimeMillis() - Duration.ofHours(24).toMillis();\ntable.expireSnapshots().expireOlderThan(watermark).commit();","handlingStrategy":"retry","validationCode":"checkState(table.io().newInputFile(manifest.path()).exists(),\n    \"delete manifest missing: \" + manifest.path());","typeGuard":null,"tryCatchPattern":"try {\n  readDeleteFiles(manifest);\n} catch (RuntimeException e) {\n  if (e.getMessage().startsWith(\"Failed to read delete manifest:\")) {\n    // retry with backoff; surface e.getCause() for storage diagnostics\n  } else throw e;\n}","preventionTips":["Pin manifest retention to outlast pipeline runtime","Grant workers read access to the table metadata location","Monitor for concurrent table-maintenance jobs"],"tags":["iceberg","delete-manifest","io"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}