istio/istio · critical

failed to load remote CA certs: %v

Error message

failed to load remote CA certs: %v

What it means

maybeCreateCA runs with ENABLE_CA=true and useRemoteCerts (a root/RA cert config was detected), so it calls loadCACerts to read the externally provided CA files from LocalCertDir; that read/validation failed. istiod wraps it as 'failed to load remote CA certs' and exits because it cannot act as CA without the external key material.

Source

Thrown at pilot/pkg/bootstrap/server.go:1262

			},
		},
		Debugger: args.KrtDebugger,
	})
	s.XDSServer.ListRemoteClusters = s.multiclusterController.ListRemoteClusters
	s.addStartFunc("multicluster controller", func(stop <-chan struct{}) error {
		return s.multiclusterController.Run(stop)
	})
}

// maybeCreateCA creates and initializes the built-in CA if needed.
func (s *Server) maybeCreateCA(caOpts *caOptions) error {
	// CA signing certificate must be created only if CA is enabled.
	if features.EnableCAServer {
		log.Info("creating CA and initializing public key")
		var err error
		if useRemoteCerts.Get() {
			if err = s.loadCACerts(caOpts, LocalCertDir.Get()); err != nil {
				return fmt.Errorf("failed to load remote CA certs: %v", err)
			}
		}
		// May return nil, if the CA is missing required configs - This is not an error.
		// This is currently only used for K8S signing.
		if caOpts.ExternalCAType != "" {
			if s.RA, err = s.createIstioRA(caOpts); err != nil {
				return fmt.Errorf("failed to create RA: %v", err)
			}
		}
		// If K8S signs - we don't need to use the built-in istio CA.
		if !s.isK8SSigning() {
			if s.CA, err = s.createIstioCA(caOpts); err != nil {
				return fmt.Errorf("failed to create CA: %v", err)
			}
		}
	}
	return nil
}

View on GitHub (pinned to 8dc789c5cf)

Solutions

1. Check the wrapped cause: not-found vs parse vs mismatch, and list the actual files in the cert dir.
2. Mount the full external CA set with the exact expected filenames (ca-cert.pem, ca-key.pem, cert-chain.pem, root-cert.pem).
3. Verify each file: openssl x509 for certs, openssl pkey for the key, and that ca-cert matches ca-key.
4. If you only wanted an external root for verification (not signing), unset the flag combination that turns on remote signing (remove ca-key.pem / disable CA server) and restart.

Example fix

# before: secret only has root-cert.pem
# after
kubectl create secret generic cacerts -n istio-system \
  --from-file=ca-cert.pem --from-file=ca-key.pem \
  --from-file=cert-chain.pem --from-file=root-cert.pem
# restart istiod with the secret mounted at /etc/cacerts

# Validate the external CA file set and key/cert match.
for f in ca-cert.pem ca-key.pem cert-chain.pem root-cert.pem; do
  test -s "/etc/cacerts/$f" || echo "missing: $f"
done
openssl x509 -in /etc/cacerts/ca-cert.pem -noout >/dev/null || echo 'bad ca-cert.pem'
openssl pkey -in /etc/cacerts/ca-key.pem -noout >/dev/null || echo 'bad ca-key.pem'

if err = s.loadCACerts(caOpts, LocalCertDir.Get()); err != nil {
    return fmt.Errorf("failed to load remote CA certs: %v", err) // names missing/unreadable file
}

When it happens

Trigger: EXTERNAL_CA / remote cert mode where /etc/cacerts (or --certDir) lacks the complete file set (ca-cert.pem, ca-key.pem, root-cert.pem, cert-chain.pem); files unreadable due to permissions; key/cert mismatch during parsing.

Common situations: Plugging istiod into an existing intermediate CA by mounting a secret with wrong keys; cert-manager/vault-provisioned secrets whose filenames do not match what loadCACerts expects; forgetting ca-key.pem when only intending root verification.

Related errors

• failed generating istiod key cert %v
• failed to create an istiod CA: %w
• failed to create an istiod CA: %v
• failed to create a self-signed istiod CA: %v
• first time load IstiodCert failed: %v

AI-assisted analysis of istio/istio@8dc789c5cf (2026-08-15). Data as JSON: /api/errors/f5cd61db8f40cd85. Report an issue: GitHub.