apache/beam · error · IOException

Interrupted while waiting before retrying insert of

Error message

Interrupted while waiting before retrying insert of

What it means

Thrown in DatasetServiceImpl.insertAll's retry loop when sleeper.sleep(nextBackoffMillis) before re-inserting failed rows is interrupted. The interrupt flag is restored and an IOException carrying the pending retryRows is thrown.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java:1336

          throw new IOException("Interrupted while inserting " + rowsToPublish);
        } catch (ExecutionException e) {
          streamingInsertsResults.updateStreamingInsertsMetrics(
              ref, rowList.size(), rowList.size());
          throw new RuntimeException(e.getCause());
        }

        if (allErrors.isEmpty()) {
          break;
        }
        long nextBackoffMillis = backoff.nextBackOffMillis();
        if (nextBackoffMillis == BackOff.STOP) {
          break;
        }
        try {
          sleeper.sleep(nextBackoffMillis);
        } catch (InterruptedException e) {
          Thread.currentThread().interrupt();
          throw new IOException("Interrupted while waiting before retrying insert of " + retryRows);
        }
        rowsToPublish = retryRows;
        idsToPublish = retryIds;
        streamingInsertsResults.updateRetriedRowsWithStatus(
            BigQuerySinkMetrics.INTERNAL, retryRows.size());
        // print first 5 failures
        int numErrorToLog = Math.min(allErrors.size(), 5);
        LOG.info(
            "Retrying {} failed inserts to BigQuery. First {} fails: {}",
            rowsToPublish.size(),
            numErrorToLog,
            allErrors.subList(0, numErrorToLog));
        allErrors.clear();
      }
      if (successfulRows != null) {
        for (int i = 0; i < rowsToPublish.size(); i++) {
          if (!failedIndices.contains(i)) {
            successfulRows.add(

View on GitHub (pinned to 12126d8942)

Solutions

Re-run the failed batch — rows in the message were not successfully inserted
Tune InsertRetryPolicy to fail fast on persistent throttling instead of long retry loops
Reduce write throughput / batch size if BigQuery quotas cause repeated 503 retries
Avoid hard shutdowns; drain the pipeline before termination

Example fix

// before
new InsertRetryPolicy().retrying(); // endless retries keep thread sleeping
// after
new InsertRetryPolicy().forInsertErrors(Arrays.asList(
    BigQueryIO.Operation.SCOPE,
    BigQueryIO.ErrorType.RETRYABLE)); // bounded retry set, fail fast otherwise

Defensive patterns

Strategy: retry

Validate before calling

// bound retries in the policy so writers don't sleep indefinitely
InsertRetryPolicy policy = InsertRetryPolicy.retryTransientErrors(); // rather than always-retry

Try / catch

try {
  pipeline.run().waitUntilFinish();
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Interrupted while waiting before retrying insert of")) {
    // re-submit these rows; they were never successfully inserted
  }
}

Prevention

Keep retry policies bounded so threads aren't in backoff sleeps during shutdown
Reduce repeated 503 throttling via batch-size/throughput tuning
Drain pipelines before termination
Re-run interrupted batches — rows in the message are unacknowledged

When it happens

Trigger: A streaming insert batch had rows that failed with retryable errors; the code sleeps before the next retry attempt and the thread is interrupted (job cancel, worker teardown).

Common situations: Cancelling a Dataflow job while rows are mid-retry; spot/preemptible worker eviction; prolonged BigQuery throttling (503 rate limits) keeping the writer in the retry loop when shutdown arrives.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8dfb8583cfe47f27. Report an issue: GitHub.