{"record":{"id":"2010d72522b173e4","repo":"apache/beam","slug":"unexpected-timeout-waiting-for-element-future-to-resolve","errorCode":null,"errorMessage":"Unexpected timeout waiting for element future to resolve after the writer was closed","messagePattern":"Unexpected timeout waiting for element future to resolve after the writer was closed","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java","lineNumber":1459,"sourceCode":"        try {\n          bigtableWriter.close();\n        } catch (IOException e) {\n          // If the writer fails due to a batching exception, but no failures were detected\n          // it means that error handling was enabled, and that errors were detected and routed\n          // to the error queue. Bigtable will successfully write other failures in the batch,\n          // so this exception should be ignored\n          if (!(e.getCause() instanceof BatchingException)) {\n            throw e;\n          }\n        }\n\n        // Sanity check: ensure that all element futures are resolved. This should be already be the\n        // case once bigtableWriter.close() finishes.\n        try {\n          CompletableFuture.allOf(outstandingWrites.toArray(new CompletableFuture<?>[0]))\n              .get(1, TimeUnit.MINUTES);\n        } catch (TimeoutException e) {\n          throw new IllegalStateException(\n              \"Unexpected timeout waiting for element future to resolve after the writer was closed\",\n              e);\n        }\n\n        if (!reportedLineage) {\n          bigtableWriter.reportLineage();\n          reportedLineage = true;\n        }\n        bigtableWriter = null;\n      }\n\n      for (KV<BigtableWriteException, BoundedWindow> badRecord : badRecords) {\n        try {\n          badRecordRouter.route(\n              c,\n              badRecord.getKey().getRecord(),\n              inputCoder,\n              (Exception) badRecord.getKey().getCause(),","sourceCodeStart":1441,"sourceCodeEnd":1477,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java#L1441-L1477","documentation":"After BigtableWriter.close() completes, the sanity check joins all outstanding element futures with a 1-minute timeout. If any future still has not resolved, an IllegalStateException('Unexpected timeout waiting for element future to resolve after the writer was closed') is thrown — an internal invariant violation indicating a write's completion callback never fired even though close() was supposed to flush everything.","triggerScenarios":"close() on BigtableWriter finishes but one or more CompletableFuture entries in outstandingWrites never complete — typically caused by a hung/failed mutation callback, a stuck Bigtable client RPC, or a callback that swallowed its completion signal.","commonSituations":"Bigtable service unavailability/slowness during sink teardown, client connection issues that leave RPCs dangling, resource exhaustion on workers (thread pool starvation) preventing callbacks from running.","solutions":["Check worker logs for the underlying Bigtable client exception on the mutation that never completed","Verify network connectivity and Bigtable service health from the runner's workers","Retry the pipeline; if reproducible, upgrade Beam and google-cloud-bigtable client versions","Reduce batch size / concurrency limits to avoid overwhelming the Bigtable client under load"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  // write to Bigtable via BigtableIO\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"Unexpected timeout waiting for element future\")) {\n    // retry the batch; check Bigtable service health and client connectivity\n  }\n  throw e;\n}","preventionTips":["Monitor Bigtable latency/error rates; back off batch sizes under saturation","Keep Beam and google-cloud-bigtable client versions current","Avoid custom writer hooks that could swallow completion callbacks"],"tags":["java","bigtable","sink","timeout","concurrency"],"backgroundTag":"request-timeout","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}