apache/beam · error

failed to open database

Error message

failed to open database: %v

What it means

queryFn.ProcessElement opens a database connection per element with sql.Open(f.Driver, f.Dsn). If the driver fails to initialize the connection, the error is wrapped as "failed to open database: %v" naming the driver. With database/sql this usually surfaces real connection problems (bad DSN, unreachable host, unknown driver) at first use.

Solutions

1. Add the blank driver import, e.g. import _ "github.com/lib/pq" (or the driver matching f.Driver).
2. Verify the DSN string by connecting with psql/mysql CLI or a small standalone Go program using the same DSN.
3. Confirm the driver name string matches the driver's registered name exactly (e.g. "postgres", "mysql", "sqlite3").
4. Check network reachability/DNS/firewall from the worker environment to the database host.

Example fix

// before
import "database/sql" // driver never registered

// after
import (
  "database/sql"
  _ "github.com/lib/pq"
)

db, err := sql.Open(driver, dsn)
if err != nil { return err }
if err := db.PingContext(ctx); err != nil { return fmt.Errorf("cannot reach %s: %w", driver, err) }
db.Close()

When it happens

Trigger: sql.Open(f.Driver, f.Dsn) returns an error: driver name not registered (missing import/blank import of the driver), or the driver immediately validates the DSN and fails.

Common situations: Typo'd driver name ("postgres" vs "pgx"); forgetting the blank import `_ "github.com/lib/pq"`; malformed DSN (wrong password format, missing host); container/network where the DB host is unreachable.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• failed to prepare query
• failed to run query
• failed to scan
• Attempting to create database
• Attempting to create database

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1f13b4ff2fd3e48e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/databaseio/database.go:75

	return beam.ParDo(s, &queryFn{Driver: driver, Dsn: dsn, Query: query, Type: beam.EncodedType{T: t}}, imp, beam.TypeDefinition{Var: beam.XType, T: t})
}

type queryFn struct {
	// Project is the project
	Driver string `json:"driver"`
	// Project is the project
	Dsn string `json:"dsn"`
	// Table is the table identifier.
	Query string `json:"query"`
	// Type is the encoded schema type.
	Type beam.EncodedType `json:"type"`
}

func (f *queryFn) ProcessElement(ctx context.Context, _ []byte, emit func(beam.X)) error {
	//TODO move DB Open and Close to Setup and Teardown methods or StartBundle and FinishBundle
	db, err := sql.Open(f.Driver, f.Dsn)
	if err != nil {
		return errors.Wrapf(err, "failed to open database: %v", f.Driver)
	}
	defer db.Close()
	statement, err := db.PrepareContext(ctx, f.Query)
	if err != nil {
		return errors.Wrapf(err, "failed to prepare query: %v", f.Query)
	}
	defer statement.Close()
	rows, err := statement.QueryContext(ctx)
	if err != nil {
		return errors.Wrapf(err, "failed to run query: %v", f.Query)
	}
	defer rows.Close()
	var mapper rowMapper
	var columns []string
	for rows.Next() {
		reflectRow := reflect.New(f.Type.T)
		row := reflectRow.Interface() // row : *T
		if mapper == nil {

View on GitHub (pinned to 12126d8942)