{"record":{"id":"13b1bda82c86c1a7","repo":"apache/beam","slug":"interrupted-while-waiting-before-retrying-insertall","errorCode":null,"errorMessage":"Interrupted while waiting before retrying insertAll","messagePattern":"Interrupted while waiting before retrying insertAll","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java","lineNumber":1092,"sourceCode":"                    .withOperation(\"insert_all\")\n                    .withFullResourceName(BigQueryHelpers.toTableFullResourceName(ref))\n                    .create()) {\n              LOG.info(\n                  \"BigQuery insertAll error, retrying: {}\",\n                  ApiErrorExtractor.INSTANCE.getErrorMessage(e));\n            }\n            try {\n              long nextBackOffMillis = backoff1.nextBackOffMillis();\n              if (nextBackOffMillis == BackOff.STOP) {\n                throw e;\n              }\n              sleeper.sleep(nextBackOffMillis);\n              totalBackoffMillis += nextBackOffMillis;\n              final long totalBackoffMillisSoFar = totalBackoffMillis;\n              maxThrottlingMsec.getAndUpdate(current -> Math.max(current, totalBackoffMillisSoFar));\n              result.updateRetriedRowsWithStatus(errorReason, rows.size());\n            } catch (InterruptedException interrupted) {\n              throw new IOException(\"Interrupted while waiting before retrying insertAll\");\n            }\n          }\n        }\n      }\n    }\n\n    @VisibleForTesting\n    <T> long insertAll(\n        TableReference ref,\n        List<FailsafeValueInSingleWindow<TableRow, TableRow>> rowList,\n        @Nullable List<String> insertIdList,\n        BackOff backoff,\n        FluentBackoff rateLimitBackoffFactory,\n        final Sleeper sleeper,\n        InsertRetryPolicy retryPolicy,\n        List<ValueInSingleWindow<T>> failedInserts,\n        ErrorContainer<T> errorContainer,\n        boolean skipInvalidRows,","sourceCodeStart":1074,"sourceCodeEnd":1110,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L1074-L1110","documentation":"Thrown inside DatasetServiceImpl.insertAll when the thread sleeping between streaming-insert retries (throttling backoff before the next insertAll attempt) is interrupted. The InterruptedException is converted to an IOException with this message so it can propagate through the Beam sink's checked-exception API.","triggerScenarios":"BigQuery streaming insert receives retryable errors (e.g. 503 rate limiting), the code sleeps via sleeper.sleep(nextBackOffMillis) before retrying, and the executing thread is interrupted (pipeline cancel, worker shutdown, or another thread interrupting).","commonSituations":"Cancelling a running Dataflow/Flink job mid-insert; worker teardown during autoscaling; user code interrupting threads around a blocking BigQuery write.","solutions":["Let the pipeline finish or cancel gracefully instead of hard-killing workers","Check for user code that interrupts executor threads doing BigQuery writes","Reduce retryable error volume (smaller batches, backoff tuning) so the code spends less time in the sleep/retry loop","Catch and inspect the IOException message to confirm interruption rather than a real insert failure"],"exampleFix":"// before\nThread.interrupt(); // somewhere in user DoFn teardown while insert in flight\n// after\n// allow in-flight writes to complete; cancel via pipeline.cancel() or await completion","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Interrupted while waiting before retrying insertAll\")) {\n    // thread was interrupted during job shutdown; treat as cancellation, not data loss\n  }\n}","preventionTips":["Don't interrupt worker threads executing BigQuery writes","Cancel jobs via pipeline.cancel() and let stages drain","Reduce throttling retries by staying within BigQuery streaming quotas","Use deterministic row IDs so re-running after interruption is idempotent"],"tags":["bigquery","gcp","threading","interruption"],"backgroundTag":"thread-interrupted","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}