kubernetes/kops · error

unable to build client-cert CA pools

Error message

unable to build client-cert CA pools

What it means

NewChallengeServer parses the CA bundle PEM into an x509 pool used to verify client certificates (mTLS). If AppendCertsFromPEM cannot parse any certificate from the bundle, server construction fails with this error. It means the caBundle bytes are not valid PEM certificates.

Source

Thrown at pkg/bootstrap/challenge_server.go:73

func NewChallengeServer(clusterName string, caBundle []byte) (*ChallengeServer, error) {
	serverCertificate, err := BuildChallengeServerCertificate(clusterName)
	if err != nil {
		return nil, err
	}
	tlsConfig := &tls.Config{
		Certificates: []tls.Certificate{*serverCertificate},
	}

	var servingCA bytes.Buffer
	for _, cert := range serverCertificate.Certificate {
		if err := pem.Encode(&servingCA, &pem.Block{Type: "CERTIFICATE", Bytes: cert}); err != nil {
			return nil, err
		}
	}

	clientCAs := x509.NewCertPool()
	if !clientCAs.AppendCertsFromPEM(caBundle) {
		return nil, fmt.Errorf("unable to build client-cert CA pools")
	}
	tlsConfig.ClientCAs = clientCAs
	tlsConfig.ClientAuth = tls.RequireAndVerifyClientCert

	return &ChallengeServer{
		RequiredSubject: challengeKopsControllerSubject(clusterName),
		tlsConfig:       tlsConfig,
		servingCA:       servingCA.Bytes(),
	}, nil
}

type Challenge struct {
	ChallengeID     string
	ChallengeSecret []byte
}

func (s *ChallengeServer) createChallenge() *Challenge {
	c := &Challenge{}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Pass the full cluster CA PEM bundle (-----BEGIN CERTIFICATE----- blocks) as caBundle
2. Decode base64 wrapping if the source stores it encoded
3. Verify each PEM block parses (openssl x509 -in / -text)
4. Re-export the CA bundle from the cluster pki directory

Example fix

// before
caBundle, _ := os.ReadFile("tls.key") // wrong file
srv, err := NewChallengeServer(..., caBundle, ...)
// after
caBundle, _ := os.ReadFile("ca.crt")
srv, err := NewChallengeServer(..., caBundle, ...)

pool := x509.NewCertPool()
if !pool.AppendCertsFromPEM(caBundle) {
	return fmt.Errorf("caBundle contains no parseable PEM certificates")
}

srv, err := NewChallengeServer(...)
if err != nil && strings.Contains(err.Error(), "unable to build client-cert CA pools") {
	// check the CA bundle file path and PEM contents before retrying
	return err
}

When it happens

Trigger: Calling NewChallengeServer (directly or via getNodeConfigFromServers/Run) with a caBundle that is empty of parseable certs: DER data, a private key, truncated PEM, or wrong file content.

Common situations: Pointing the server at the wrong PEM file (e.g. server cert chain without CAs); base64 not decoded before passing; templating artifact truncating the bundle; using the node CA instead of the cluster CA that signed client certs.

Related errors

• error loading certificate pool
• parsing key: %v
• error reading client keypair: %v
• failed to verify client certificate chain: %w
• failed to create certificate: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/33e36e2b9c39c08b. Report an issue: GitHub.