apache/beam · error · RuntimeException

More than %d attempts to call AppendRows failed. Last encoun

Error message

More than %d attempts to call AppendRows failed. Last encountered error: %s. Please check if the destination table exists and if the service account has the bigquery.tables.updateData permission.

What it means

RuntimeException thrown by the WriteRecordsDoFn in the sharded Storage Write path after all configured attempts to call AppendRows failed. The message carries the last underlying error; PERMISSION_DENIED or NOT_FOUND statuses add a hint to check the destination table's existence and the service account's bigquery.tables.updateData permission.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWritesShardedRecords.java:1086

          initializeContexts.accept(contexts);
          try {
            retryManager.run(true);
          } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw e;
          } catch (Exception e) {
            Status.Code statusCode = Status.fromThrowable(e).getCode();
            String errorMessage =
                String.format(
                    "More than %d attempts to call AppendRows failed. Last encountered error: %s",
                    maxRetries, e.toString());
            if (statusCode == Status.Code.PERMISSION_DENIED
                || statusCode == Status.Code.NOT_FOUND) {
              errorMessage +=
                  ". Please check if the destination table exists and if the service account has the "
                      + "bigquery.tables.updateData permission.";
            }
            throw new RuntimeException(errorMessage, e);
          }

          appendSplitDistribution.update(numAppends);
          if (autoUpdateSchema) {
            @Nullable StreamAppendClient streamAppendClient =
                appendClientHolder.getStreamAppendClient();
            TableSchema originalSchema = appendClientHolder.get().getTableSchema();

            @Nullable TableSchema updatedSchemaReturned =
                (streamAppendClient != null) ? streamAppendClient.getUpdatedSchema() : null;
            // Update the table schema and clear the append client.
            if (updatedSchemaReturned != null) {
              Optional<TableSchema> newSchema =
                  TableSchemaUpdateUtils.getUpdatedSchema(originalSchema, updatedSchemaReturned);
              if (newSchema.isPresent()) {
                APPEND_CLIENTS.invalidate(messageConverters.getAppendClientKey(element.getKey()));
                LOG.debug(
                    "Fetched updated schema for table {}:\n\t{}", tableId, updatedSchemaReturned);

View on GitHub (pinned to 12126d8942)

Solutions

Inspect the wrapped cause for the final gRPC status code
Grant bigquery.tables.updateData on the destination table to the pipeline service account
Verify the table exists for the full pipeline duration
Check BigQuery quotas and service health; restart from checkpoint after transient failures
Align the writer schema with the table or enable auto schema update

Example fix

// before
// pipeline SA lacks write access -> NOT_FOUND/PERMISSION_DENIED after retries
// after
// grant bigquery.dataEditor on the dataset to the SA running the pipeline

Defensive patterns

Strategy: retry

Validate before calling

com.google.cloud.bigquery.Table t = bigquery.getTable(tableId);
if (t == null) throw new IllegalStateException("Destination table missing: " + tableId);
// ensure the pipeline SA has bigquery.tables.updateData (roles/bigquery.dataEditor)

Try / catch

try {
  pipeline.run().waitUntilFinish();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("attempts to call AppendRows failed")) {
    log.error("AppendRows retries exhausted; cause:", e.getCause());
    // grant permissions / restore table / wait out outage, then resubmit
  } else { throw e; }
}

Prevention

Grant roles/bigquery.dataEditor on the dataset to the pipeline service account
Keep destination tables alive for the pipeline's lifetime
Enable schema auto-update or keep writer and table schemas in sync
Alert on BigQuery quota/service issues before they exhaust AppendRows retries

When it happens

Trigger: AppendRows fails on every retry for a shard's records — persistent gRPC errors such as PERMISSION_DENIED, NOT_FOUND, unavailability, or unrecoverable schema problems — until the retry counter exceeds the attempt limit.

Common situations: Missing bigquery.tables.updateData for the pipeline service account; destination table dropped mid-run; Storage Write API outage or quota limits; schema changed on the table so buffered rows no longer validate.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/925ebb151fbc1d8d. Report an issue: GitHub.