What causes "unable to partition query: %v" in apache/beam?

Inspect the formatted Spanner error in the panic message and fix the SQL.

unable to partition query: %v

Error message

unable to partition query: %v

What it means

generatePartitionsFn panics when the Spanner PartitionQueryWithOptions call fails while trying to split the query into parallel partitions. The client error is formatted into the panic message. This typically indicates a bad SQL statement or a query that Spanner cannot partition.

Source

Thrown at sdks/go/pkg/beam/io/spannerio/generate_partitions.go:92

	return &generatePartitionsFn{
		spannerFn: newSpannerFn(db),
		Query:     query,
		Options:   options,
	}
}

func (f *generatePartitionsFn) ProcessElement(ctx context.Context, _ []byte, emit func(partitionedRead)) error {
	txn, err := f.client.BatchReadOnlyTransaction(ctx, f.Options.TimestampBound)
	if err != nil {
		panic("unable to create batch read only transaction: " + err.Error())
	}
	defer txn.Close()

	mode := spannerpb.ExecuteSqlRequest_PROFILE

	partitions, err := txn.PartitionQueryWithOptions(ctx, spanner.Statement{SQL: f.Query}, partitionOptions(f.Options), spanner.QueryOptions{Mode: &mode})
	if err != nil {
		panic(fmt.Sprintf("unable to partition query: %v", err))
	}

	for _, p := range partitions {
		emit(newPartitionedRead(txn.ID, p))
	}

	return nil
}

View on GitHub (pinned to 12126d8942)

Solutions

Inspect the formatted Spanner error in the panic message and fix the SQL.
Test the query directly with gcloud or the Spanner Studio to confirm it is valid and partitionable.
Simplify to a full-table scan or add an appropriate index for partitioned reads.
Verify the tables/columns exist in the target database (schema drift check).

Example fix

// before
query := "SELECT * FROM userss" // typo
// after
query := "SELECT * FROM users"

Defensive patterns

Strategy: try-catch

Validate before calling

// pre-flight: run the query outside Beam to confirm validity/partitionability
_, err := client.Single().Query(ctx, spanner.Statement{SQL: query}).Next()
if err != nil {
    return fmt.Errorf("query invalid: %w", err)
}

Try / catch

defer func() {
    if r := recover(); r != nil {
        if s, ok := r.(string); ok && strings.Contains(s, "unable to partition query") {
            // log s, fix SQL, restart pipeline
        } else { panic(r) }
    }
}()

Prevention

Test the exact SQL against the target database before launching the pipeline.
Avoid LIMIT-without-ORDER-BY and other constructs Spanner cannot partition.
Watch for schema drift; validate table/column names in CI.
Keep the query simple (single-table scan) for best partitionability.

When it happens

Trigger: txn.PartitionQueryWithOptions returns an error: invalid SQL syntax, referencing a nonexistent table/column, query not partitionable (e.g. no suitable index / LIMIT without ORDER BY constraints), or permission issues.

Common situations: Typos in the query string; schema drift after the query was written; using a query with constructs Spanner disallows for partitioning; IAM restrictions on the worker.

Understand the failure class

Background: "query failed", "%w: SQL error" — wrapped database query errors in Go libraries explained — this error's family across 3 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/331906e3a6e059e1. Report an issue: GitHub.