temporalio/temporal · critical

failed to load tls x509 key pair: %v

Error message

failed to load tls x509 key pair: %v

What it means

This error is returned by registerTLSConfig in the MySQL SQL plugin when tls.LoadX509KeyPair fails to read or parse the client certificate and key files configured via cfg.TLS.CertFile and cfg.TLS.KeyFile. It wraps the underlying error (file-not-found, parse failure, or cert/key mismatch) so the DB connection setup fails fast during createConnection. It only fires when TLS is enabled and both a cert file and key file are configured.

Source

Thrown at common/persistence/sql/sqlplugin/mysql/session/session.go:231

		rootCertPool := x509.NewCertPool()
		pem, err := os.ReadFile(cfg.TLS.CaFile)
		if err != nil {
			return fmt.Errorf("failed to load CA files: %v", err)
		}
		if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
			return fmt.Errorf("failed to append CA file")
		}
		tlsConfig.RootCAs = rootCertPool
	}

	if cfg.TLS.CertFile != "" && cfg.TLS.KeyFile != "" {
		clientCert := make([]tls.Certificate, 0, 1)
		certs, err := tls.LoadX509KeyPair(
			cfg.TLS.CertFile,
			cfg.TLS.KeyFile,
		)
		if err != nil {
			return fmt.Errorf("failed to load tls x509 key pair: %v", err)
		}
		clientCert = append(clientCert, certs)
		tlsConfig.Certificates = clientCert
	}

	// In order to use the TLS configuration you need to register it. Once registered you use it by specifying
	// `tls` in the connect attributes.
	err := mysql.RegisterTLSConfig(customTLSName, tlsConfig)
	if err != nil {
		return fmt.Errorf("failed to register tls config: %v", err)
	}

	if cfg.ConnectAttributes == nil {
		cfg.ConnectAttributes = map[string]string{}
	}

	// If no `tls` connect attribute is provided then we override it to our newly registered tls config automatically.
	// This allows users to simply provide a tls config without needing to remember to also set the connect attribute

View on GitHub (pinned to bde624efd1)

Solutions

1. Verify cfg.TLS.CertFile and cfg.TLS.KeyFile point to existing, readable PEM files on the host (ls/permissions as the DB-using process user).
2. Confirm cert and key match: compare modulus/hash of certificate public key against the private key (e.g. openssl x509 -noout -modem vs openssl rsa -noout -modulus).
3. Re-read the error text appended by %v: 'no such file or directory' means path issue; 'failed to find any PEM data' means wrong/empty content; 'tls: private key does not match' means pairing issue.
4. Regenerate the client cert/key pair from the same CA as the MySQL server and redeploy both files together.
5. If mTLS client auth is not required, unset TLS.CertFile/TLS.KeyFile (keep TLS.Enabled for server-side TLS only).

Example fix

// before
tls:
  enabled: true
  certFile: /etc/temporal/tls/client.crt
  keyFile: /etc/temporal/tls/client.key

// after (verify files exist and match, e.g. via secret mount)
tls:
  enabled: true
  certFile: /etc/temporal/tls/mysql-client.crt
  keyFile: /etc/temporal/tls/mysql-client.key

// Go: validate before constructing SQL config
for _, p := range []string{cfg.TLS.CertFile, cfg.TLS.KeyFile} {
    f, err := os.Open(p)
    if err != nil {
        return fmt.Errorf("TLS file unreadable %q: %w", p, err)
    }
    f.Close()
}
if _, err := tls.LoadX509KeyPair(cfg.TLS.CertFile, cfg.TLS.KeyFile); err != nil {
    return fmt.Errorf("cert/key pair invalid: %w", err)
}

if err := persistenceFactory.NewSQLDB(...); err != nil {
    if strings.Contains(err.Error(), "failed to load tls x509 key pair") {
        logger.Fatal("mTLS client cert/key invalid; check TLS.CertFile/TLS.KeyFile", tag.Error(err))
    }
    return err
}

When it happens

Trigger: createConnection -> registerTLSConfig with cfg.TLS.Enabled=true and both cfg.TLS.CertFile and cfg.TLS.KeyFile set to non-empty paths, where tls.LoadX509KeyPair returns an error: nonexistent path, unreadable file, invalid PEM, wrong file type (e.g. cert passed as key), or key not matching the certificate.

Common situations: Mounting secrets in Kubernetes with wrong paths; cert/key files rotated or deleted before process start; deploying config that enables mTLS against a MySQL server without provisioning client certs; mismatched cert/key pair after re-issuing one side; copy-pasted config from another host.

Understand the failure class

• SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

• failed to load CA files: %v
• failed to append CA file
• failed to register tls config: %v
• only one of keyData or keyFile properties should be specifie
• only one of caData or caFile properties should be specified

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/d51de3b5e36f7ab8. Report an issue: GitHub.