{"record":{"id":"8dfb8583cfe47f27","repo":"apache/beam","slug":"interrupted-while-waiting-before-retrying-insert-of","errorCode":null,"errorMessage":"Interrupted while waiting before retrying insert of ","messagePattern":"Interrupted while waiting before retrying insert of ","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java","lineNumber":1336,"sourceCode":"          throw new IOException(\"Interrupted while inserting \" + rowsToPublish);\n        } catch (ExecutionException e) {\n          streamingInsertsResults.updateStreamingInsertsMetrics(\n              ref, rowList.size(), rowList.size());\n          throw new RuntimeException(e.getCause());\n        }\n\n        if (allErrors.isEmpty()) {\n          break;\n        }\n        long nextBackoffMillis = backoff.nextBackOffMillis();\n        if (nextBackoffMillis == BackOff.STOP) {\n          break;\n        }\n        try {\n          sleeper.sleep(nextBackoffMillis);\n        } catch (InterruptedException e) {\n          Thread.currentThread().interrupt();\n          throw new IOException(\"Interrupted while waiting before retrying insert of \" + retryRows);\n        }\n        rowsToPublish = retryRows;\n        idsToPublish = retryIds;\n        streamingInsertsResults.updateRetriedRowsWithStatus(\n            BigQuerySinkMetrics.INTERNAL, retryRows.size());\n        // print first 5 failures\n        int numErrorToLog = Math.min(allErrors.size(), 5);\n        LOG.info(\n            \"Retrying {} failed inserts to BigQuery. First {} fails: {}\",\n            rowsToPublish.size(),\n            numErrorToLog,\n            allErrors.subList(0, numErrorToLog));\n        allErrors.clear();\n      }\n      if (successfulRows != null) {\n        for (int i = 0; i < rowsToPublish.size(); i++) {\n          if (!failedIndices.contains(i)) {\n            successfulRows.add(","sourceCodeStart":1318,"sourceCodeEnd":1354,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L1318-L1354","documentation":"Thrown in DatasetServiceImpl.insertAll's retry loop when sleeper.sleep(nextBackoffMillis) before re-inserting failed rows is interrupted. The interrupt flag is restored and an IOException carrying the pending retryRows is thrown.","triggerScenarios":"A streaming insert batch had rows that failed with retryable errors; the code sleeps before the next retry attempt and the thread is interrupted (job cancel, worker teardown).","commonSituations":"Cancelling a Dataflow job while rows are mid-retry; spot/preemptible worker eviction; prolonged BigQuery throttling (503 rate limits) keeping the writer in the retry loop when shutdown arrives.","solutions":["Re-run the failed batch — rows in the message were not successfully inserted","Tune InsertRetryPolicy to fail fast on persistent throttling instead of long retry loops","Reduce write throughput / batch size if BigQuery quotas cause repeated 503 retries","Avoid hard shutdowns; drain the pipeline before termination"],"exampleFix":"// before\nnew InsertRetryPolicy().retrying(); // endless retries keep thread sleeping\n// after\nnew InsertRetryPolicy().forInsertErrors(Arrays.asList(\n    BigQueryIO.Operation.SCOPE,\n    BigQueryIO.ErrorType.RETRYABLE)); // bounded retry set, fail fast otherwise","handlingStrategy":"retry","validationCode":"// bound retries in the policy so writers don't sleep indefinitely\nInsertRetryPolicy policy = InsertRetryPolicy.retryTransientErrors(); // rather than always-retry","typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Interrupted while waiting before retrying insert of\")) {\n    // re-submit these rows; they were never successfully inserted\n  }\n}","preventionTips":["Keep retry policies bounded so threads aren't in backoff sleeps during shutdown","Reduce repeated 503 throttling via batch-size/throughput tuning","Drain pipelines before termination","Re-run interrupted batches — rows in the message are unacknowledged"],"tags":["bigquery","gcp","threading","retry"],"backgroundTag":"thread-interrupted","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}