{"record":{"id":"5bb177df4992ca11","repo":"apache/beam","slug":"expected-at-least-one-overlapping-task-in-bidirectional-list","errorCode":null,"errorMessage":"Expected at least one overlapping task in bidirectional list","messagePattern":"Expected at least one overlapping task in bidirectional list","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/cdc/ChangelogScanner.java","lineNumber":639,"sourceCode":"\n      StructLike globalDeleteLower = null;\n      StructLike globalDeleteUpper = null;\n      for (TaskAndBounds t : deleteTasks) {\n        if (t.overlaps) {\n          if (globalDeleteLower == null || idComp.compare(t.lowerId, globalDeleteLower) < 0) {\n            globalDeleteLower = t.lowerId;\n          }\n          if (globalDeleteUpper == null || idComp.compare(t.upperId, globalDeleteUpper) > 0) {\n            globalDeleteUpper = t.upperId;\n          }\n        }\n      }\n\n      if (globalInsertLower == null\n          || globalDeleteLower == null\n          || globalInsertUpper == null\n          || globalDeleteUpper == null) {\n        throw new IllegalStateException(\n            \"Expected at least one overlapping task in bidirectional list\");\n      }\n\n      overlapLower =\n          idComp.compare(globalInsertLower, globalDeleteLower) > 0\n              ? globalInsertLower\n              : globalDeleteLower;\n      overlapUpper =\n          idComp.compare(globalInsertUpper, globalDeleteUpper) < 0\n              ? globalInsertUpper\n              : globalDeleteUpper;\n    }\n\n    return new AnalysisResult(unidirectional, bidirectional, overlapLower, overlapUpper);\n  }\n\n  /**\n   * Routes bi-directional tasks from an {@link AnalysisResult} to either the in-memory local","sourceCodeStart":621,"sourceCodeEnd":657,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/cdc/ChangelogScanner.java#L621-L657","documentation":"During bidirectional changelog analysis, analyzeFiles computes global lower/upper record-identifier bounds over the overlapping insert and delete task lists. If any of the four global bounds is null, the assumed invariant — that both lists are non-empty and overlap — was violated, so it throws IllegalStateException rather than producing bogus bounds.","triggerScenarios":"analyzeFiles reaches the bidirectional branch with either the insert or delete task list empty (no overlapping tasks), so globalInsertLower/DeleteLower/Upper never get assigned, then the null check throws.","commonSituations":"A CDC scan window where only inserts or only deletes occur but the code path expects bidirectional overlap; boundary/boundary-metric edge cases in file sequence ordering; a bug triggered by unusual table states (e.g. all data deleted in one snapshot).","solutions":["Report this to the Beam maintainers with table snapshot details — it indicates a classification/overlap bug, not user misconfiguration.","Work around by restructuring the scan interval so insert and delete tasks genuinely overlap (e.g. adjust snapshot range/start strategy).","Ensure the table's changelog scan isn't split into degenerate windows with only one task kind (compact/delete old snapshots to normalize file metadata).","Verify sequence-number ordering in table metadata is sane (repair via Iceberg metadata validation tools) since bounds derive from file sequence bounds."],"exampleFix":"null","handlingStrategy":"try-catch","validationCode":"// pre-check that both task lists are non-empty before bidirectional analysis\nif (insertTasks.isEmpty() || deleteTasks.isEmpty()) {\n  throw new IllegalStateException(\"Bidirectional analysis requires insert and delete tasks\");\n}","typeGuard":"boolean hasBothSides(List<ChangelogScanTask> tasks) {\n  boolean ins = tasks.stream().anyMatch(t -> t instanceof AddedRowsScanTask);\n  boolean del = tasks.stream().anyMatch(t -> t instanceof DeletedDataFileScanTask || t instanceof DeletedRowsScanTask);\n  return ins && del;\n}","tryCatchPattern":"try {\n  AnalysisResult r = scanner.result(tasks);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"Expected at least one overlapping task\")) {\n    // fall back to conservative allBidirectional analysis\n  } else { throw e; }\n}","preventionTips":["Report occurrences upstream — indicates a scanner bug, not user error.","Avoid degenerate scan windows containing only one task kind.","Normalize table state via compaction/snapshot expiration before CDC scans."],"tags":["java","iceberg","cdc","internal-invariant","scanner"],"backgroundTag":"internal-invariant-violation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}