apache/beam · error · RuntimeException

More than %d attempts to call AppendRows failed. Last encoun

Error message

More than %d attempts to call AppendRows failed. Last encountered error: %s. Please check if the destination table exists and if the service account has the bigquery.tables.updateData permission.

What it means

RuntimeException thrown by StorageApiWriteUnshardedRecords.flush after all configured attempts to call the Storage Write API AppendRows RPC failed. The message includes the last gRPC error; for PERMISSION_DENIED or NOT_FOUND statuses an extra hint about the table's existence and bigquery.tables.updateData permission is appended.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java:804

                invalidateAppendClient(true);
                allowedRetry = 5;
              } else {
                allowedRetry = 35;
              }

              // Maximum number of times we retry before we fail the work item.
              if (failedContext.failureCount > allowedRetry) {
                String errorMessage =
                    String.format(
                        "More than %d attempts to call AppendRows failed. Last encountered error: %s",
                        allowedRetry, error != null ? error.toString() : "unknown");
                if (statusCode == Status.Code.PERMISSION_DENIED
                    || statusCode == Status.Code.NOT_FOUND) {
                  errorMessage +=
                      ". Please check if the destination table exists and if the service account has the "
                          + "bigquery.tables.updateData permission.";
                }
                throw new RuntimeException(errorMessage, error);
              }

              // The following errors are known to be persistent, so always fail the work item in
              // this case.
              if (statusCode.equals(Status.Code.OUT_OF_RANGE)
                  || statusCode.equals(Status.Code.ALREADY_EXISTS)) {
                throw new RuntimeException(
                    "Append to stream "
                        + this.streamName
                        + " failed with invalid "
                        + "offset of "
                        + failedContext.offset);
              }

              // Schema mismatched exceptions can happen if the table was recently updated. Since
              // vortex caches schemas
              // we might see the new schema before vortex does. In this case, we simply need to
              // retry.

View on GitHub (pinned to 12126d8942)

Solutions

Read the wrapped cause and error message for the final gRPC status code
Grant the pipeline service account the bigquery.tables.updateData permission on the destination table
Confirm the destination table exists and wasn't deleted or renamed during the run
Check BigQuery quotas and Storage Write API health; retry the pipeline after transient outages
If schema changed on the table, update the writer schema or enable schema auto-update

Example fix

// before: service account has read-only access
// after: grant write permission
bq update --source project:dataset dataset  # then in IAM:
# roles/bigquery.dataEditor on the dataset for the pipeline SA

Defensive patterns

Strategy: retry

Validate before calling

com.google.cloud.bigquery.Table t = bigquery.getTable(tableId);
if (t == null) throw new IllegalStateException("Destination table missing");
// check SA permission
Policy p = bigquery.getIamPolicy(tableId.getTableId().getDataset());
// ensure the pipeline SA has roles/bigquery.dataEditor

Try / catch

try {
  pipeline.run().waitUntilFinish();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("attempts to call AppendRows failed")) {
    log.error("AppendRows exhausted retries; cause:", e.getCause());
    // fix permission/table issue indicated by final status, then resubmit
  } else { throw e; }
}

Prevention

Grant the pipeline SA bigquery.tables.updateData (e.g. roles/bigquery.dataEditor) on the dataset
Monitor table deletions during the run; avoid dropping tables under active pipelines
Watch BigQuery quota usage for the Storage Write API
Tune retry/backoff parameters for known transient outages

When it happens

Trigger: AppendRows fails on every retry attempt for a batch of rows — e.g. persistent PERMISSION_DENIED or NOT_FOUND statuses, gRPC unavailability, or schema mismatch — until the attempt counter exceeds the retry limit.

Common situations: Service account lacking bigquery.tables.updateData on the destination table; table/stream deleted while the pipeline runs; BigQuery Storage Write API outage or quota exhaustion; write stream closed/finalized unexpectedly.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5c9278ac49a9e744. Report an issue: GitHub.