apache/beam · error · SpannerException

SpannerException (rethrown from operation failure/timeout…

Error message

SpannerException (rethrown from operation failure/timeout cause)

What it means

PartitionMetadataAdminDao.createPartitionMetadataTable issues a Cloud Spanner admin operation and blocks on its result. If the operation fails or times out, the underlying SpannerException cause is rethrown so callers see a typed Spanner error; if there is no cause, the exception is wrapped via SpannerExceptionFactory. An InterruptedException is converted to an interrupted SpannerException without swallowing the interrupt flag.

Solutions

1. Inspect the rethrown SpannerException's code (e.g. RESOURCE_EXHAUSTED, DEADLINE_EXCEEDED, PERMISSION_DENIED) and fix the corresponding Spanner-side issue.
2. Retry the table creation; DDL operations are idempotent if you check for ALREADY_EXISTS.
3. Verify the service account has spanner.admin / database creation IAM roles.
4. Check Spanner instance health/quota if timeouts recur.

// Pre-check IAM and instance before creating the table
boolean canCreate = spanner.getDatabaseAdminClient()
    .listDatabases(instanceId).getValues().stream()
    .noneMatch(db -> db.getId().getDatabase().equals(databaseId));

if (!(e.getCause() instanceof SpannerException)) { e = SpannerExceptionFactory.asSpannerException(e); }

try { dao.createPartitionMetadataTable(); } catch (SpannerException e) {
  if (e.getErrorCode() == ErrorCode.ALREADY_EXISTS) { /* ok */ }
  else if (e.isRetryable()) { backoffRetry(); }
  else { throw e; }
}

When it happens

Trigger: Calling createPartitionMetadataTable when the Spanner admin CreateDatabase/UpdateDatabaseDdl operation fails, exceeds TIMEOUT_MINUTES (5 minutes), or the thread is interrupted while waiting on op.get().

Common situations: Spanner database out of quota or DDL timeout; transient Spanner outages during pipeline bootstrap; permissions missing for spanner.databases.create; slow schema update queuing on a busy database.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• unable to create batch read only transaction:
• BulkMutation took too long to close
• could not fetch database schema
• Error while parsing the DataChangeRecord
• Failed to fetch partition_mode for change stream

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/87e5b09ad767e069. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/dao/PartitionMetadataAdminDao.java:255

              + " ON "
              + names.getTableName()
              + " ("
              + COLUMN_CREATED_AT
              + ","
              + COLUMN_START_TIMESTAMP
              + ")");
    }
    OperationFuture<Void, UpdateDatabaseDdlMetadata> op =
        databaseAdminClient.updateDatabaseDdl(instanceId, databaseId, ddl, null);
    try {
      // Initiate the request which returns an OperationFuture.
      op.get(TIMEOUT_MINUTES, TimeUnit.MINUTES);
    } catch (ExecutionException | TimeoutException e) {
      // If the operation failed or timed out during execution, expose the cause.
      if (e.getCause() != null) {
        throw (SpannerException) e.getCause();
      } else {
        throw SpannerExceptionFactory.asSpannerException(e);
      }
    } catch (InterruptedException e) {
      // Throw when a thread is waiting, sleeping, or otherwise occupied,
      // and the thread is interrupted, either before or during the activity.
      throw SpannerExceptionFactory.propagateInterrupt(e);
    }
  }

  /**
   * Drops the metadata table. This operation should complete in {@link
   * PartitionMetadataAdminDao#TIMEOUT_MINUTES} minutes.
   */
  public void deletePartitionMetadataTable(List<String> indexes) {
    List<String> ddl = new ArrayList<>();
    if (this.isPostgres()) {
      indexes.forEach(index -> ddl.add("DROP INDEX \"" + index + "\""));
      ddl.add("DROP TABLE \"" + names.getTableName() + "\"");
    } else {

View on GitHub (pinned to 12126d8942)