thanos-io/thanos · error

reading system certificate pool

Error message

reading system certificate pool

What it means

NewClientConfig wraps this error when no custom caCert is given and x509.SystemCertPool() fails to load the operating system's trusted root certificate pool. This typically means no system CA store is available or readable.

Solutions

1. Install system CA certs: apt-get install -y ca-certificates or apk add ca-certificates.
2. Use a distroless/base image variant that includes CA certificates.
3. Explicitly provide caCert path to NewClientConfig instead of relying on the system pool.
4. Unset/fix SSL_CERT_FILE / SSL_CERT_DIR if they point to missing locations.

Example fix

# before
FROM scratch
COPY app /app
# after
FROM gcr.io/distroless/static:latest
COPY app /app

if caCert == "" {
    if _, err := x509.SystemCertPool(); err != nil {
        return fmt.Errorf("no system CA pool: %w; install ca-certificates or set caCert", err)
    }
}

cfg, err := tls.NewClientConfig(logger, cert, key, "", serverName, skipVerify, ver)
if err != nil && strings.Contains(err.Error(), "reading system certificate pool") {
    // fall back to an explicitly provided CA bundle
    cfg, err = tls.NewClientConfig(logger, cert, key, "/etc/ssl/certs/ca-certificates.crt", serverName, skipVerify, ver)
}

When it happens

Trigger: NewClientConfig called with empty caCert on a system without the expected CA bundle (e.g. a scratch/alpine/distroless container lacking /etc/ssl/certs or ca-certificates package).

Common situations: Minimal Docker images without ca-certificates installed; running on Windows with restricted crypto API; SSL_CERT_FILE/SSL_CERT_DIR pointing at nonexistent paths (Go 1.18+ returns error).

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

• SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

• setup gRPC server
• setup gRPC server
• both server key and certificate must be provided
• server credentials
• building client CA

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/8c317ea4e9f67d8c. Report an issue: GitHub.

Appendix: source

Thrown at pkg/tls/options.go:150

// minTLSVersion must be one of 1.0, 1.1, 1.2, 1.3 per GetTlsVersion().
func NewClientConfig(logger log.Logger, cert, key, caCert, serverName string, skipVerify bool, minTLSVersion string) (*tls.Config, error) {
	var certPool *x509.CertPool
	if caCert != "" {
		caPEM, err := os.ReadFile(filepath.Clean(caCert))
		if err != nil {
			return nil, errors.Wrap(err, "reading client CA")
		}

		certPool = x509.NewCertPool()
		if !certPool.AppendCertsFromPEM(caPEM) {
			return nil, errors.Wrap(err, "building client CA")
		}
		level.Debug(logger).Log("msg", "TLS client using provided certificate pool")
	} else {
		var err error
		certPool, err = x509.SystemCertPool()
		if err != nil {
			return nil, errors.Wrap(err, "reading system certificate pool")
		}
		level.Debug(logger).Log("msg", "TLS client using system certificate pool")
	}

	var (
		mtlsVersion uint16
		err         error
	)

	if minTLSVersion != "" {
		mtlsVersion, err = GetTlsVersion(minTLSVersion)
		if err != nil {
			return nil, err
		}
		level.Debug(logger).Log("msg", fmt.Sprintf("setting minimum TLS version to %s", minTLSVersion))
	}
	tlsCfg := &tls.Config{
		RootCAs:    certPool,

View on GitHub (pinned to 35b8b99117)