JuliusBrussee/caveman · error

postgres: begin tenant schema inspection

Error message

postgres: begin tenant schema inspection: %w

What it means

ValidateTenantSchema opens a REPEATABLE READ, READ ONLY transaction to inspect tenant table structure. If the transaction cannot be started, the pool's underlying connection error is wrapped with this message.

Solutions

1. Inspect the wrapped %w error for the root cause (connection refused, auth failed, too many connections) and address it.
2. Confirm the database is reachable: `pg_isready -h <host> -p <port>` from the app environment.
3. Check pgxpool configuration limits (MaxConns) against the server's max_connections and pgbouncer pool_size.
4. Add readiness gating/retry with backoff before calling NewRuntimePool at startup.

Example fix

// before
pool, err := postgresconfig.NewRuntimePool(ctx, os.Getenv("DATABASE_URL"), "tenant_role")
// immediately on container start
// after
waitForDB(ctx, os.Getenv("DATABASE_URL")) // poll with pg_isready/backoff
pool, err := postgresconfig.NewRuntimePool(ctx, os.Getenv("DATABASE_URL"), "tenant_role")

if err := pool.Ping(ctx); err != nil {
	return fmt.Errorf("database not reachable before schema validation: %w", err)
}

err := postgresconfig.ValidateTenantSchema(ctx, pool)
if err != nil {
	var pgErr *pgconn.PgError
	if !errors.As(err, &pgErr) { // likely connection-level; retry with backoff
		return retryWithBackoff(ctx, func() error { return postgresconfig.ValidateTenantSchema(ctx, pool) })
	}
	return err
}

When it happens

Trigger: NewRuntimePool calls ValidateTenantSchema; pool.BeginTx fails because the pool has no healthy connections — database unreachable, credentials rejected, max connections exhausted, or TLS negotiation failure.

Common situations: Postgres not yet ready at container startup; wrong host/port in the database URL; too many clients (max_connections); pgbouncer rejecting new connections; TLS verification failures with the hardened config.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• postgres: finish tenant schema inspection
• postgres: inspect runtime identity
• postgres: inspect tenant foreign keys
• postgres: inspect tenant tables
• postgres: parse DATABASE_URL

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/c9518afe4dfb5b7a. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/postgresconfig/postgresconfig.go:232

// organizationPolicyExpression matches the only accepted tenant predicate:
// organization_id equals the transaction-local tenant GUC, optionally cast to
// the column's scalar type by PostgreSQL. It is anchored so OR TRUE, a different
// operator, a different GUC, or an extra predicate cannot pass by containing
// the two expected strings.
var organizationPolicyExpression = regexp.MustCompile(`^organization_id=\(*current_setting\('app\.current_organization_id'(?:::text)?(?:,true)?\)\)*(?:::[A-Za-z_][A-Za-z0-9_.$"]*)?$`)

// ValidateTenantSchema proves structural isolation invariants that are easy to
// regress during schema evolution. Every tenant table must fail closed under
// FORCE RLS, and every foreign-key edge between tenant tables must carry the
// shared organization scope (plus project scope when both tables have it).
func ValidateTenantSchema(ctx context.Context, pool *pgxpool.Pool) error {
	tx, err := pool.BeginTx(ctx, pgx.TxOptions{
		IsoLevel:   pgx.RepeatableRead,
		AccessMode: pgx.ReadOnly,
	})
	if err != nil {
		return fmt.Errorf("postgres: begin tenant schema inspection: %w", err)
	}
	defer tx.Rollback(ctx)

	tables, err := inspectTenantTables(ctx, tx)
	if err != nil {
		return fmt.Errorf("postgres: inspect tenant tables: %w", err)
	}
	foreignKeys, err := inspectTenantForeignKeys(ctx, tx)
	if err != nil {
		return fmt.Errorf("postgres: inspect tenant foreign keys: %w", err)
	}
	if violations := tenantSchemaViolations(tables, foreignKeys); len(violations) > 0 {
		return fmt.Errorf("postgres: tenant schema isolation incomplete: %s", strings.Join(violations, "; "))
	}
	if err := validateResolverSchema(ctx, tx); err != nil {
		return err
	}
	if err := tx.Commit(ctx); err != nil {

View on GitHub (pinned to 3ee70a1026)