{"record":{"id":"182ae9f59ee00591","repo":"egametang/ET","slug":"service-discovery-request-retry-exhausted-request","errorCode":null,"errorMessage":"service discovery request retry exhausted, request: {requestName}","messagePattern":"service discovery request retry exhausted, request: (.+?)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"Packages/cn.etetet.servicediscovery/Scripts/Hotfix/Server/ServiceDiscoveryAgentSystem.cs","lineNumber":671,"sourceCode":"\n                    bool shouldResolveMaster = self.ShouldResolveMasterAfterFailure(e);\n                    if (shouldResolveMaster)\n                    {\n                        self.InvalidateEndpointAndTriggerBackgroundRegister(\"call-failure\");\n                    }\n                    else\n                    {\n                        Log.Warning(\n                            $\"ServiceDiscoveryAgent call failed without endpoint fallback scene: {self.Root().Name} request: {requestName} target: {targetActorId} error: {e.Message}\");\n                        throw;\n                    }\n                }\n\n                int delay = GetMasterResolveRetryDelay(attempt);\n                await self.Root().TimerComponent.WaitAsync(delay);\n            }\n\n            throw new Exception($\"service discovery request retry exhausted, request: {requestName}\");\n        }\n\n        private static bool ShouldResolveMasterAfterFailure(this ServiceDiscoveryAgent self, Exception e)\n        {\n\n            if (e is RpcException rpcException)\n            {\n                return ServiceDiscoveryErrorHelper.ShouldTriggerFailover(rpcException.Error);\n            }\n\n            return true;\n        }\n\n        private static int GetMasterResolveRetryDelay(int retry)\n        {\n            if (retry <= 0)\n            {\n                return MasterResolveRetryBaseInterval;","sourceCodeStart":653,"sourceCodeEnd":689,"githubUrl":"https://github.com/egametang/ET/blob/5cab01f7a8bee5f49f4781eebe9e2b1c6d7ebe0f/Packages/cn.etetet.servicediscovery/Scripts/Hotfix/Server/ServiceDiscoveryAgentSystem.cs#L653-L689","documentation":"Thrown at line 671 in ForwardToDiscoveryByFailoverCore after all 4 failover attempts (maxAttemptCount) failed. Each attempt either threw (and was failover-retried) or hit an endpoint problem. The retry budget is exhausted, so the request is given up as a hard failure.","triggerScenarios":"Four consecutive attempts failed: each attempt's exception either triggered InvalidateEndpointAndTriggerBackgroundRegister + backoff, or the endpoint could not be resolved. Sustained master instability, repeated follower rejections, or repeated message timeouts across all attempts.","commonSituations":"Master flapping during a network partition; repeated ERR_MessageTimeout because the master is overloaded; a failover loop where each re-resolve lands on a different (also failing) master; chronic DB unavailability causing every register/response to fail.","solutions":["Stabilize the master scene (resource, network, election) so at least one of 4 attempts succeeds.","Increase maxAttemptCount only if the failure mode is genuinely transient and slow to recover (prefer fixing the master).","Check logs for the per-attempt errors (the catch block logs warnings) to find the dominant failure code.","Verify master lease/election health and DB connectivity."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"if (!agent.IsReady())\n{\n    // do not enter a failover loop destined to exhaust\n    return;\n}","typeGuard":null,"tryCatchPattern":"catch (Exception e) when (e.Message.Contains(\"retry exhausted\"))\n{\n    Log.Error($\"discovery failover exhausted: {e.Message}\");\n    // escalate: page on-call, circuit-break the operation\n}","preventionTips":["Stabilize the master so failover succeeds within the budget.","Watch per-attempt warning logs to find the dominant failure code.","Tune maxAttemptCount only after fixing root cause."],"tags":["csharp","et-framework","service-discovery","rpc","retry","master-election","network"],"backgroundTag":null,"analyzedSha":"5cab01f7a8bee5f49f4781eebe9e2b1c6d7ebe0f","analyzedAt":"2026-08-13T21:10:40.377Z","schemaVersion":2},"datasetVersion":"2026-08-14T00:17:13.853Z"}