apache/beam · critical

unable to create batch read only transaction:

Error message

unable to create batch read only transaction:

What it means

During processing, generatePartitionsFn panics if it cannot create a Cloud Spanner batch read-only transaction. The error from the Spanner client is appended to the panic message. Unlike construction-time panics, this fires on workers while the pipeline is running, usually due to connectivity, credentials, or an invalid database path.

Source

Thrown at sdks/go/pkg/beam/io/spannerio/generate_partitions.go:84

	db string,
	query string,
	options queryOptions,
) *generatePartitionsFn {
	if db == "" {
		panic("no database provided")
	}

	return &generatePartitionsFn{
		spannerFn: newSpannerFn(db),
		Query:     query,
		Options:   options,
	}
}

func (f *generatePartitionsFn) ProcessElement(ctx context.Context, _ []byte, emit func(partitionedRead)) error {
	txn, err := f.client.BatchReadOnlyTransaction(ctx, f.Options.TimestampBound)
	if err != nil {
		panic("unable to create batch read only transaction: " + err.Error())
	}
	defer txn.Close()

	mode := spannerpb.ExecuteSqlRequest_PROFILE

	partitions, err := txn.PartitionQueryWithOptions(ctx, spanner.Statement{SQL: f.Query}, partitionOptions(f.Options), spanner.QueryOptions{Mode: &mode})
	if err != nil {
		panic(fmt.Sprintf("unable to partition query: %v", err))
	}

	for _, p := range partitions {
		emit(newPartitionedRead(txn.ID, p))
	}

	return nil
}

View on GitHub (pinned to 12126d8942)

Solutions

  1. Read the wrapped error in the panic message and fix the underlying Spanner client error.
  2. Verify the database path and that the pipeline's service account has Spanner read permissions.
  3. Check network connectivity (VPC, firewall, TestEndpoint override) from workers.
  4. Confirm the TimestampBound options are valid (e.g. a readable timestamp not in the future).

Example fix

// before
spannerio.Read with TestEndpoint pointing at a dead emulator
// after
use a reachable emulator endpoint and export SPANNER_EMULATOR_HOST, or remove the override for production
Defensive patterns

Strategy: try-catch

Validate before calling

// pre-flight: open a plain client and ping
client, err := spanner.NewClient(ctx, db)
if err != nil {
    return fmt.Errorf("spanner unreachable: %w", err)
}
client.Close()

Try / catch

defer func() {
    if r := recover(); r != nil {
        if s, ok := r.(string); ok && strings.Contains(s, "unable to create batch read only transaction") {
            // log s, alert, retry pipeline with backoff
        } else { panic(r) }
    }
}()

Prevention

When it happens

Trigger: f.client.BatchReadOnlyTransaction(ctx, f.Options.TimestampBound) returns an error — e.g. unreachable Spanner endpoint, missing/insufficient IAM permissions (spanner.backup/reader or db reader), bad TestEndpoint override, or invalid TimestampBound.

Common situations: Running Dataflow workers without access to the Spanner VPC/network; service account lacking spanner.databaseClient or viewer roles; typo'd instance/database path; stale or invalid credentials.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/afd0271e6304bf84. Report an issue: GitHub.