{"record":{"id":"90b23443cc489580","repo":"apache/beam","slug":"failed-to-plan-deleted-rows-tasks","errorCode":null,"errorMessage":"Failed to plan deleted rows tasks","messagePattern":"Failed to plan deleted rows tasks","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/iceberg/BeamBaseIncrementalChangelogScan.java","lineNumber":687,"sourceCode":"                  return ResidualEvaluator.of(spec, residualFilter, isCaseSensitive());\n                });\n\n        tasks.add(\n            new BaseDeletedRowsScanTask(\n                changeOrdinal,\n                snapshot.snapshotId(),\n                dataFile.copy(shouldKeepStats()),\n                addedDeletes,\n                existingDeletes,\n                schemaString,\n                specString,\n                residuals));\n\n        // Mark this file as processed for this snapshot\n        alreadyProcessedPaths.add(filePath);\n      }\n    } catch (Exception e) {\n      throw new RuntimeException(\"Failed to plan deleted rows tasks\", e);\n    }\n  }\n\n  private boolean shouldKeepStats() {\n    Set<Integer> columns = columnsToKeepStats();\n    return columns != null && !columns.isEmpty();\n  }\n\n  /**\n   * Loads delete files from manifests by parsing each manifest.\n   *\n   * @param manifests the delete manifests to load\n   * @return list of delete files\n   */\n  private Iterable<DeleteFile> loadDeleteFiles(\n      List<ManifestFile> manifests, Long targetSnapshotId) {\n    Queue<DeleteFile> allDeleteFiles = new ConcurrentLinkedQueue<>();\n","sourceCodeStart":669,"sourceCodeEnd":705,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/iceberg/BeamBaseIncrementalChangelogScan.java#L669-L705","documentation":"processSnapshotForDeletedRowsTasks plans tasks for rows deleted via delete files in a snapshot. Exceptions encountered while reading delete manifests or building deleted-row tasks are wrapped into this RuntimeException.","triggerScenarios":"Calling planDeletedRowsTasks when a delete manifest entry cannot be read, its delete file is missing, or task construction (residual evaluation, schema/spec resolution) fails.","commonSituations":"Delete files orphaned by concurrent compaction; object-store I/O errors while opening manifests; incompatible delete-file format after an Iceberg upgrade.","solutions":["Inspect the wrapped cause to find which manifest or delete file failed and verify its availability","Retry the pipeline — transient storage errors are the most frequent cause","Check for concurrent table maintenance (compaction, expiration) racing with the changelog scan and coordinate/serialize those operations"],"exampleFix":"// before\ntable.newIncrementalChangelogScan() ... // run concurrently with rewriteDataFiles/compaction\n// after\n// pause table maintenance jobs, or run the scan against a snapshot range untouched by compaction\nscan = table.newIncrementalChangelogScan().fromSnapshotId(preCompactionSnap);","handlingStrategy":"retry","validationCode":"// preflight: read all delete manifests in range\nfor (ManifestFile m : deleteManifestsInRange) {\n  try (FileIO io = table.io()) {\n    io.newInputFile(m.path()).exists();\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  planDeletedRowsTasks(...);\n} catch (RuntimeException e) {\n  if (e.getMessage().equals(\"Failed to plan deleted rows tasks\")) {\n    // inspect e.getCause(); retry transient storage failures\n  } else throw e;\n}","preventionTips":["Serialize compaction/rewrite jobs with the changelog scan","Enable object-store client retry policies on workers","Alert on orphaned delete files after failed maintenance"],"tags":["iceberg","delete-files","io"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}