apache/beam · critical

failed to initialise Spanner client

Error message

failed to initialise Spanner client: %v

What it means

spannerio.Setup creates a Cloud Spanner client during pipeline setup; this error wraps any failure from spanner.NewClient (or its underlying spanner.NewClientWithConfig path). Since Setup runs once per worker before processing, a failure here stops the pipeline at startup with the underlying Spanner error appended.

Solutions

1. Read the wrapped %v cause and fix accordingly: usually credentials (run `gcloud auth application-default login` or provide service-account creds to workers).
2. Verify the database string exactly matches projects/<project>/instances/<instance>/databases/<db> and that the instance/database exist.
3. Enable the Spanner Admin/API on the GCP project and confirm the service account has roles/spanner.databaseUser.
4. Check network egress from the runner/workers to spanner.googleapis.com:443 (VPC firewall, NAT config).
5. Retry after fixing — the client is created in Setup, so any fix requires restarting the pipeline.

Example fix

// before
f.Database = "my-instance/my-db" // wrong format
// failed to initialise Spanner client: ...

// after
f.Database = "projects/my-project/instances/my-instance/databases/my-db"

// Pre-flight checks before launching the pipeline:
if !strings.HasPrefix(db, "projects/") {
    return fmt.Errorf("invalid database path: %s", db)
}
creds, err := google.FindDefaultCredentials(ctx)
if err != nil {
    return fmt.Errorf("no GCP credentials found: %w", err)
}
_ = creds

When it happens

Trigger: Running a Beam pipeline with the spannerio.Read/Write transform when spanner.NewClient fails: invalid database path format, missing/insufficient GCP credentials, project/database/instance does not exist, network egress blocked, or client creation timeout.

Common situations: Missing GOOGLE_APPLICATION_CREDENTIALS or default credentials on the worker; typo in projects/p/instances/i/databases/d path; Spanner API not enabled on the project; private-network runners with no route to spanner.googleapis.com.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• could not create data operations client
• Could not resolve GCP project ID for secret
• Could not resolve GCP project ID
• Error creating Data Catalog client
• Error while parsing the DataChangeRecord

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/65ea8c0b2137f2e7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/spannerio/common.go:63

}

func (f *spannerFn) Setup(ctx context.Context) error {
	if f.client == nil {
		var opts []option.ClientOption

		// Append emulator options assuming endpoint is local (for testing).
		if f.TestEndpoint != "" {
			opts = []option.ClientOption{
				option.WithEndpoint(f.TestEndpoint),
				option.WithGRPCDialOption(grpc.WithTransportCredentials(insecure.NewCredentials())),
				option.WithoutAuthentication(),
				internaloption.SkipDialSettingsValidation(),
			}
		}

		client, err := spanner.NewClient(ctx, f.Database, opts...)
		if err != nil {
			return fmt.Errorf("failed to initialise Spanner client: %v", err)
		}

		f.client = client
	}

	return nil
}

func (f *spannerFn) Teardown() {
	if f.client != nil {
		f.client.Close()
	}
}

View on GitHub (pinned to 12126d8942)