{"record":{"id":"b9dd6095db788d05","repo":"cilium/cilium","slug":"cep-deleted-externally-or-significant-informer-del","errorCode":null,"errorMessage":"CEP deleted externally or significant informer delay","messagePattern":"CEP deleted externally or significant informer delay","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"pkg/endpointmanager/endpointsynchronizer.go","lineNumber":230,"sourceCode":"\t\t\t\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\t\t\t\tscopedLog.Debug(\"Error getting indexed CiliumEndpoint from store\", logfields.Error, err)\n\t\t\t\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t\t\t\tfor _, cep := range objs {\n\t\t\t\t\t\t\t\t\t\t\tif cep.Namespace == cepOwner.GetNamespace() && cep.Name == cepName {\n\t\t\t\t\t\t\t\t\t\t\t\tcepExists = true\n\t\t\t\t\t\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif !cepExists && ctx.Err() == nil {\n\t\t\t\t\t\t\tscopedLog.Warn(\"CEP was deleted externally or there is a significant delay on the CEP informer, will recreate on next iteration\")\n\t\t\t\t\t\t\tneedInit = true\n\t\t\t\t\t\t\tlocalCEP = nil\n\t\t\t\t\t\t\treturn fmt.Errorf(\"CEP deleted externally or significant informer delay\")\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tscopedLog.Debug(\"Skipping CiliumEndpoint update because it has not changed\")\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\n\t\t\t\tif needInit {\n\t\t\t\t\tstate := e.GetState()\n\t\t\t\t\t// Don't bother to create if the\n\t\t\t\t\t// endpoint is already disconnecting\n\t\t\t\t\tif state == endpoint.StateDisconnecting ||\n\t\t\t\t\t\tstate == endpoint.StateDisconnected {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\n\t\t\t\t\tscopedLog.Debug(\"Getting CEP during an initialization\")\n\t\t\t\t\tif firstTry {\n\t\t\t\t\t\t// First we try getting CEP from the API server cache, as it's cheaper.","sourceCodeStart":212,"sourceCodeEnd":248,"githubUrl":"https://github.com/cilium/cilium/blob/ac7b90affa4baf0642e6685319d56907b3a73a6d/pkg/endpointmanager/endpointsynchronizer.go#L212-L248","documentation":"During CiliumEndpoint update, the synchronizer checks that the CEP still exists in the local informer cache before applying changes. If it does not exist while the context is still live, it concludes the CEP was deleted externally (or the informer is lagging) and returns this error to force a full re-init of the CEP on the next iteration. This is a deliberate error signal used to reset the sync state machine, not an unexpected crash.","triggerScenarios":"Another actor (kubectl delete, CEP owner change, operator, or another agent after a CEP ownership handoff) deleted the CEP, or the CEP informer cache lags behind apiserver state while the agent attempts an update.","commonSituations":"Manual kubectl delete ciliumendpoint; race with agent restart/migration where the new agent deleted the old CEP; informer lag under heavy apiserver load; node under memory pressure slowing watch delivery.","solutions":["Usually benign: the controller recreates/re-initializes the CEP on the next iteration — verify the CEP reappears (kubectl get cep)","If persistent, check apiserver/informer health (watch latency, apiserver metrics) and agent CPU throttling","Look for external controllers or scripts deleting CEPs and remove/fix them","Confirm no CEP ownership conflicts between agents on the same node"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// confirm the CEP actually exists before assuming external deletion\n_, err := clientset.CiliumV2().CiliumEndpoints(ns).Get(ctx, name, metav1.GetOptions{})\nif apierrors.IsNotFound(err) {\n\tlog.Info(\"CEP confirmed deleted; recreation expected on next iteration\")\n}","typeGuard":null,"tryCatchPattern":"if err := updateCEP(ctx, cep); err != nil {\n\tif strings.Contains(err.Error(), \"CEP deleted externally\") {\n\t\tlog.Warn(\"CEP missing from cache; letting next iteration re-init it\")\n\t\treturn\n\t}\n\treturn err\n}","preventionTips":["Don't delete CEPs manually; delete the Pod and let Cilium reconcile","Monitor apiserver watch latency to rule out informer lag","Avoid running competing controllers that manage CEPs","Verify agent restart/migration sequences complete cleanly"],"tags":["cilium","kubernetes","cep","informer"],"backgroundTag":"object-deleted-externally","analyzedSha":"ac7b90affa4baf0642e6685319d56907b3a73a6d","analyzedAt":"2026-08-31T18:27:15.868Z","schemaVersion":2},"datasetVersion":"2026-08-31T19:17:28.585Z"}