{"record":{"id":"a6a6e433c2b30ab2","repo":"apache/shenyu","slug":"renew-master-fail-s","errorCode":null,"errorMessage":"renew master fail, %s","messagePattern":"renew master fail, (.+?)","errorType":"exception","errorClass":"ShenyuException","httpStatus":null,"severity":"critical","filePath":"shenyu-admin/src/main/java/org/apache/shenyu/admin/mode/cluster/service/ShenyuClusterService.java","lineNumber":111,"sourceCode":"            while (renewed) {\n                // sleeps selectPeriod seconds then renew the lock\n                TimeUnit.SECONDS.sleep(clusterProperties.getSelectPeriod());\n                \n                renewed = shenyuClusterSelectMasterService.checkMasterStatus();\n                if (renewed) {\n                    if (LOG.isDebugEnabled()) {\n                        LOG.debug(\"renew master success\");\n                    }\n                }\n            }\n        } catch (Exception e) {\n            LOG.error(\"select master error\", e);\n            // close the upstream check service\n            upstreamCheckService.close();\n            instanceCheckService.close();\n            \n            String message = String.format(\"renew master fail, %s\", e.getMessage());\n            throw new ShenyuException(message);\n        } finally {\n            try {\n                shenyuClusterSelectMasterService.releaseMaster();\n            } catch (Exception e) {\n                LOG.error(\"release master error\", e);\n            }\n        }\n    }\n    \n    @Override\n    public void start(final String host, final int port, final String contextPath) {\n        startSelectMasterTask(host, String.valueOf(port), contextPath);\n    }\n    \n    @Override\n    public void shutdown() {\n    \n    }","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/apache/shenyu/blob/567142e07261b3e615ae8850b30f4421f455cc5d/shenyu-admin/src/main/java/org/apache/shenyu/admin/mode/cluster/service/ShenyuClusterService.java#L93-L129","documentation":"ShenyuClusterService.doSelectMaster runs the periodic task where each admin node tries to acquire/renew the distributed master lock. If any exception occurs during master selection, the service closes upstream/instance check services and rethrows as ShenyuException(\"renew master fail, <cause message>\"). The wrapped cause is the real problem.","triggerScenarios":"The scheduled startSelectMasterTask fails while acquiring the master lock — zookeeper/etcd connection loss, session expiry, lock node deleted concurrently, or any KeeperException/cluster-store error during doSelectMaster.","commonSituations":"Zookeeper session expired under load; network blip between admin nodes and the cluster store; multiple admin nodes fighting after a store restart; cluster store (zk/etcd) quorum loss.","solutions":["Read the wrapped `%s` cause in the log line — fix the underlying zookeeper/etcd error (connectivity, auth, session timeout).","Verify the cluster store is reachable and quorum-healthy from every admin node.","Increase session/connection timeouts if renewals fail under GC pauses or network latency.","Restart the failed admin instance if its check services were closed — it will not process data until it rejoins the cluster.","Ensure clocks and versions are consistent across admin nodes to avoid concurrent lock conflicts."],"exampleFix":"// before\nshenyu:\n  cluster:\n    zookeeper:\n      sessionTimeout: 3000\n// after (tolerate brief GC/network pauses)\nshenyu:\n  cluster:\n    zookeeper:\n      sessionTimeout: 60000","handlingStrategy":"retry","validationCode":"if (!clusterClient.isConnected())\n    throw new IllegalStateException(\"cluster store disconnected — master renewal will fail\");","typeGuard":null,"tryCatchPattern":"try {\n    shenyuClusterService.startSelectMasterTask();\n} catch (ShenyuException e) {\n    LOG.error(\"master renewal failed, cause: {}\", e.getCause(), e);\n    // alert: this admin node closed check services and must rejoin\n}","preventionTips":["Monitor the distributed lock path health in zookeeper/etcd","Increase session timeouts to survive GC pauses","Alert on 'renew master fail' — the node drops out of cluster duty","Keep all admin nodes on the same version and NTP-synced clocks"],"tags":["cluster","master-election","zookeeper","distributed-lock"],"backgroundTag":"invalid-state-transition","analyzedSha":"567142e07261b3e615ae8850b30f4421f455cc5d","analyzedAt":"2026-09-12T10:08:21.293Z","contentChangedAt":"2026-09-12T10:08:21.293Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}