{"record":{"id":"e0ab6d35b3cc4eda","repo":"apache/beam","slug":"ignore-error-at-closing-residualsource","errorCode":null,"errorMessage":"Ignore error at closing ResidualSource","messagePattern":"Ignore error at closing ResidualSource","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/UnboundedReadFromBoundedSource.java","lineNumber":311,"sourceCode":"        this.options = checkNotNull(options, \"options\");\n        this.done = false;\n      }\n\n      private void init(\n          @Nullable List<TimestampedValue<T>> residualElementsList,\n          @Nullable BoundedSource<T> residualSource,\n          PipelineOptions options) {\n        this.residualElements =\n            residualElementsList == null\n                ? new ResidualElements(Collections.emptyList())\n                : new ResidualElements(residualElementsList);\n\n        if (this.residualSource != null) {\n          // close current residualSource to avoid leak of reader.close() in ResidualSource\n          try {\n            this.residualSource.close();\n          } catch (IOException e) {\n            LOG.warn(\"Ignore error at closing ResidualSource\", e);\n          }\n        }\n        this.residualSource =\n            residualSource == null ? null : new ResidualSource(residualSource, options);\n      }\n\n      @Override\n      public boolean start() throws IOException {\n        return advance();\n      }\n\n      @Override\n      public boolean advance() throws IOException {\n        if (residualElements.advance()) {\n          return true;\n        } else if (residualSource != null && residualSource.advance()) {\n          return true;\n        } else {","sourceCodeStart":293,"sourceCodeEnd":329,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/UnboundedReadFromBoundedSource.java#L293-L329","documentation":"When a BoundedSource reader in UnboundedReadFromBoundedSource transitions to a new residual source, the currently held residualSource is explicitly closed to avoid leaking the underlying reader. An IOException raised during that close is only logged at WARN level and swallowed, so pipeline execution continues. This is intentional best-effort cleanup, not a failure of the read itself.","triggerScenarios":"Calling setResidualSource/getCheckpointMark on the UnboundedSourceReader when the underlying ResidualSource's reader.close() throws IOException, e.g. because the wrapped bounded source's reader failed while releasing resources (file handles, HTTP connections to a test pipeline service).","commonSituations":"Checkpointing during a drain or failover of a test-stream/bounded-to-unbounded conversion; backend or filesystem flakiness while closing the bounded reader; shutdown races where the channel behind the reader is already closed.","solutions":["Inspect the full stack trace in the log to find the underlying IOException thrown by the wrapped reader's close() and fix that root cause.","Upgrade the Beam version; the residual-source close handling has been hardened over time.","If the warning is noisy, confirm the pipeline result is still correct — the warning is deliberately non-fatal.","Ensure the wrapped BoundedSource reader tolerates being closed after the split (idempotent close)."],"exampleFix":"// before\ntry {\n  this.residualSource.close();\n} catch (IOException e) {\n  LOG.warn(\"Ignore error at closing ResidualSource\", e);\n}\n// after\ntry {\n  this.residualSource.close();\n} catch (IOException e) {\n  LOG.warn(\"Ignore error at closing ResidualSource\", e);\n  // optionally count metrics / alert:\n  closeFailureCount.inc();\n}","handlingStrategy":"try-catch","validationCode":"// Before enabling BoundedSource->Unbounded conversion, verify the wrapped source closes cleanly:\ntry (BoundedSource.BoundedReader<?> r = boundedSource.createReader(options)) {\n  // advance/start once to allocate resources, then close in try-with-resources\n} // IOException on close here predicts the warning","typeGuard":null,"tryCatchPattern":"try {\n  reader.close();\n} catch (IOException e) {\n  LOG.warn(\"Ignore error at closing ResidualSource\", e); // non-fatal; continue\n}","preventionTips":["Wrap bounded sources whose readers implement idempotent close()","Watch logs for repeated close failures as a sign of underlying source bugs","Run the source in a test pipeline to catch close-time IOExceptions early","Keep Beam updated for improved residual-source lifecycle handling"],"tags":["io","java","resource-cleanup","checkpointing"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}