xds: CertificateProvider to fetch trusted roots is missing,

Error message

xds: CertificateProvider to fetch trusted roots is missing, cannot perform TLS handshake. Please check configuration on the management server

What it means

Returned by HandshakeInfo.clientSideTLSConfigInternal (internal/credentials/xds/handshake_info.go:209) when hi.rootProvider == nil. On the client side the root provider (trusted CA roots used to verify the server) is mandatory — identity provider is optional for plain TLS vs mTLS. A nil root provider means the xDS management server's security config did not supply a validation context / trusted roots for this cluster, so the client cannot safely verify the peer and aborts the TLS handshake.

Source

Thrown at internal/credentials/xds/handshake_info.go:209

// GetSANMatchersForTesting returns the SAN matchers stored in HandshakeInfo.
// To be used only for testing purposes.
func (hi *HandshakeInfo) GetSANMatchersForTesting() []matcher.StringMatcher {
	return append([]matcher.StringMatcher{}, hi.sanMatchers...)
}

// clientSideTLSConfigInternal constructs a tls.Config to be used in a
// client-side handshake based on the contents of the HandshakeInfo.
//
// hostname is passed as a parameter here instead of being part of the
// HandshakeInfo because HandshakeInfo contains cluster-level security
// configuration that applies to all endpoints in the cluster, while hostname is
// specific to each endpoint. This allows sharing a single HandshakeInfo
// instance across multiple endpoints in the same cluster.
func (hi *HandshakeInfo) clientSideTLSConfigInternal(ctx context.Context, hostname string) (*tls.Config, error) {
	// On the client side, rootProvider is mandatory. IdentityProvider is
	// optional based on whether the client is doing TLS or mTLS.
	if hi.rootProvider == nil {
		return nil, errors.New("xds: CertificateProvider to fetch trusted roots is missing, cannot perform TLS handshake. Please check configuration on the management server")
	}

	// InsecureSkipVerify needs to be set to true because we need to perform
	// custom verification to check the SAN on the received certificate.
	// Currently the Go stdlib does complete verification of the cert (which
	// includes hostname verification) or none. We are forced to go with the
	// latter and perform the normal cert validation ourselves.
	cfg := &tls.Config{
		InsecureSkipVerify: true,
		NextProtos:         []string{"h2"},
	}

	km, err := hi.rootProvider.KeyMaterial(ctx)
	if err != nil {
		return nil, fmt.Errorf("xds: fetching trusted roots from CertificateProvider failed: %v", err)
	}
	cfg.RootCAs = km.Roots

View on GitHub (pinned to 03255a9237)

Solutions

Inspect the xDS management server config for the affected cluster and add a validation context with trusted roots (validation_context.trusted_ca / ca_certificate_provider_instance).
Confirm the certprovider plugin referenced by the config is registered in the client binary (e.g. import _ the pemfile/google default plugins).
If no server-side TLS is intended for that cluster, allow fallback credentials instead of forcing xDS TLS.
Check the management server logs / xDS debug dump to see the actual DownstreamTlsContext being sent.

Example fix

// Conceptual: the fix is on the xDS control plane, not in Go code.
// before (Envoy LDS/CDS): client side lacks validation_context
// downstream_tls_context:
//   common_tls_context:
//     tls_certificates: [{...}]   # identity only, no validation_context

// after
// downstream_tls_context:
//   common_tls_context:
//     tls_certificates: [{...}]
//     validation_context:
//       trusted_ca: {filename: "/etc/grpc/root-ca.pem"}

Defensive patterns

Strategy: validation

Validate before calling

// Mostly a control-plane concern; on the client you can detect missing
// roots early by attempting one TLS handshake and treating the sentinel as
// a config error. There is no client-side fix — validate operational config:
// 1. Ensure the certprovider plugin referenced by xDS is imported.
//    import _ "google.golang.org/grpc/credentials/tls/certprovider/pemfile"
// 2. On the management server, confirm validation_context is set for the cluster.

Try / catch

// Handshake errors surface from grpc.Dial/Invoke. Retry is not useful until
// the control plane is fixed; log the sentinel and surface an actionable alert.
if err != nil && strings.Contains(err.Error(), "CertificateProvider to fetch trusted roots is missing") {
    alertOps("xDS control plane missing trusted roots for cluster")
}

Prevention

Import the certprovider plugins your xDS config references (blank import).
Validate the management server's DownstreamTlsContext includes a validation context in staging.
Set up xDS config monitoring/alerting on security-resource absence.

When it happens

Trigger: An xDS-enabled client performs a TLS handshake for a cluster whose DownstreamTlsContext lacks a validation context (no trusted_ca / root cert provider). Surfaced at handshake time via ClientSideTLSConfig -> clientSideTLSConfigInternal, returned from ClientHandshake.

Common situations: Management server (Istio/Envoy/Google Traffic Director) misconfiguration: the Listener/Cluster security policy omits the validation context; partial xDS config push where the CDS/LDS resources are inconsistent; using xDS creds on a target whose cluster has no security config and fallback was not used; certificate provider plugin failing to register.

Understand the failure class

SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

AI-assisted analysis of grpc/grpc-go@03255a9237 (2026-08-07). Data as JSON: /api/errors/dd9e5ad818f5f0e4. Report an issue: GitHub.