{"record":{"id":"f5cd61db8f40cd85","repo":"istio/istio","slug":"failed-to-load-remote-ca-certs-v","errorCode":null,"errorMessage":"failed to load remote CA certs: %v","messagePattern":"failed to load remote CA certs: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"pilot/pkg/bootstrap/server.go","lineNumber":1262,"sourceCode":"\t\t\t},\n\t\t},\n\t\tDebugger: args.KrtDebugger,\n\t})\n\ts.XDSServer.ListRemoteClusters = s.multiclusterController.ListRemoteClusters\n\ts.addStartFunc(\"multicluster controller\", func(stop <-chan struct{}) error {\n\t\treturn s.multiclusterController.Run(stop)\n\t})\n}\n\n// maybeCreateCA creates and initializes the built-in CA if needed.\nfunc (s *Server) maybeCreateCA(caOpts *caOptions) error {\n\t// CA signing certificate must be created only if CA is enabled.\n\tif features.EnableCAServer {\n\t\tlog.Info(\"creating CA and initializing public key\")\n\t\tvar err error\n\t\tif useRemoteCerts.Get() {\n\t\t\tif err = s.loadCACerts(caOpts, LocalCertDir.Get()); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to load remote CA certs: %v\", err)\n\t\t\t}\n\t\t}\n\t\t// May return nil, if the CA is missing required configs - This is not an error.\n\t\t// This is currently only used for K8S signing.\n\t\tif caOpts.ExternalCAType != \"\" {\n\t\t\tif s.RA, err = s.createIstioRA(caOpts); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create RA: %v\", err)\n\t\t\t}\n\t\t}\n\t\t// If K8S signs - we don't need to use the built-in istio CA.\n\t\tif !s.isK8SSigning() {\n\t\t\tif s.CA, err = s.createIstioCA(caOpts); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create CA: %v\", err)\n\t\t\t}\n\t\t}\n\t}\n\treturn nil\n}","sourceCodeStart":1244,"sourceCodeEnd":1280,"githubUrl":"https://github.com/istio/istio/blob/8dc789c5cf17517c64e3c36cb3288230f149dfae/pilot/pkg/bootstrap/server.go#L1244-L1280","documentation":"maybeCreateCA runs with ENABLE_CA=true and useRemoteCerts (a root/RA cert config was detected), so it calls loadCACerts to read the externally provided CA files from LocalCertDir; that read/validation failed. istiod wraps it as 'failed to load remote CA certs' and exits because it cannot act as CA without the external key material.","triggerScenarios":"EXTERNAL_CA / remote cert mode where /etc/cacerts (or --certDir) lacks the complete file set (ca-cert.pem, ca-key.pem, root-cert.pem, cert-chain.pem); files unreadable due to permissions; key/cert mismatch during parsing.","commonSituations":"Plugging istiod into an existing intermediate CA by mounting a secret with wrong keys; cert-manager/vault-provisioned secrets whose filenames do not match what loadCACerts expects; forgetting ca-key.pem when only intending root verification.","solutions":["Check the wrapped cause: not-found vs parse vs mismatch, and list the actual files in the cert dir.","Mount the full external CA set with the exact expected filenames (ca-cert.pem, ca-key.pem, cert-chain.pem, root-cert.pem).","Verify each file: openssl x509 for certs, openssl pkey for the key, and that ca-cert matches ca-key.","If you only wanted an external root for verification (not signing), unset the flag combination that turns on remote signing (remove ca-key.pem / disable CA server) and restart."],"exampleFix":"# before: secret only has root-cert.pem\n# after\nkubectl create secret generic cacerts -n istio-system \\\n  --from-file=ca-cert.pem --from-file=ca-key.pem \\\n  --from-file=cert-chain.pem --from-file=root-cert.pem\n# restart istiod with the secret mounted at /etc/cacerts","handlingStrategy":"validation","validationCode":"# Validate the external CA file set and key/cert match.\nfor f in ca-cert.pem ca-key.pem cert-chain.pem root-cert.pem; do\n  test -s \"/etc/cacerts/$f\" || echo \"missing: $f\"\ndone\nopenssl x509 -in /etc/cacerts/ca-cert.pem -noout >/dev/null || echo 'bad ca-cert.pem'\nopenssl pkey -in /etc/cacerts/ca-key.pem -noout >/dev/null || echo 'bad ca-key.pem'","typeGuard":null,"tryCatchPattern":"if err = s.loadCACerts(caOpts, LocalCertDir.Get()); err != nil {\n    return fmt.Errorf(\"failed to load remote CA certs: %v\", err) // names missing/unreadable file\n}","preventionTips":["Provision external CA secrets via automation that checks the full file set.","Match key and cert before mounting (compare modulus or use cert-manager renewals atomically).","If not running istiod as a CA, disable ENABLE_CA so remote cert loading is skipped."],"tags":["istiod","ca","pki","external-ca","certificates","startup"],"backgroundTag":null,"analyzedSha":"8dc789c5cf17517c64e3c36cb3288230f149dfae","analyzedAt":"2026-08-15T15:16:55.434Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}