{"record":{"id":"d75adc55c2019913","repo":"apache/skywalking","slug":"admin-server-failed-to-start-grpc-server","errorCode":null,"errorMessage":"admin-server: failed to start gRPC server","messagePattern":"admin-server: failed to start gRPC server","errorType":"exception","errorClass":"ModuleStartException","httpStatus":null,"severity":"critical","filePath":"oap-server/server-admin/admin-server/src/main/java/org/apache/skywalking/oap/server/admin/server/module/AdminServerModuleProvider.java","lineNumber":222,"sourceCode":"        // already registered in prepare() with a lazy ClusterNodesQuery\n        // supplier; nothing else to do here.\n    }\n\n    @Override\n    public void notifyAfterCompleted() throws ModuleStartException {\n        if (RunningMode.isInitMode()) {\n            return;\n        }\n        try {\n            if (grpcServer != null) {\n                grpcServer.start();\n                log.info(\"admin-server gRPC listening on {}:{} (peer-to-peer admin RPCs only — \"\n                             + \"MUST be reachable between OAP nodes; MUST NOT be exposed to the agent network \"\n                             + \"or operators).\",\n                         moduleConfig.getGRPCHost(), moduleConfig.getGRPCPort());\n            }\n        } catch (final ServerException e) {\n            throw new ModuleStartException(\"admin-server: failed to start gRPC server\", e);\n        }\n        if (peerChannelManager != null) {\n            peerChannelManager.start();\n        }\n        if (httpServer != null) {\n            httpServer.start();\n            log.info(\n                \"admin-server HTTP listening on {}:{} (no built-in authentication — \"\n                    + \"gateway-protect with an IP allow-list and authenticating reverse proxy; \"\n                    + \"never expose to the public internet).\",\n                moduleConfig.getHost(), moduleConfig.getPort()\n            );\n        }\n    }\n\n    @Override\n    public String[] requiredModules() {\n        return new String[] {","sourceCodeStart":204,"sourceCodeEnd":240,"githubUrl":"https://github.com/apache/skywalking/blob/102af09b4a56064e22050dded10e2c52e490d040/oap-server/server-admin/admin-server/src/main/java/org/apache/skywalking/oap/server/admin/server/module/AdminServerModuleProvider.java#L204-L240","documentation":"AdminServerModuleProvider.start() calls grpcServer.start() for the admin-internal gRPC bus; a ServerException (typically bind failure: port already in use, or missing permission to bind a privileged port) is wrapped in ModuleStartException. This happens after config validation, at actual socket bind time, and stops OAP startup.","triggerScenarios":"Another process (or a second OAP instance on the same host, or a misconfigured pod with two replicas sharing network) already listens on the configured gRPCPort; or the port is below 1024 and the process lacks CAP_NET_BIND_SERVICE.","commonSituations":"Running two OAP nodes with hostNetwork on the same machine; port collision between admin gRPCPort and another service; container restart where the old process still holds the port (TIME_WAIT/liveness overlap); Kubernetes NodePort/hostPort conflicts.","solutions":["Identify the holder of the port: ss -ltnp | grep <port> or netstat -ltnp","Free the port or move admin-server gRPCPort to an unused one, then restart","If binding a port < 1024, run with adequate capability or choose a high port","For flapping containers, ensure previous pods are fully terminated before replacements start"],"exampleFix":"# before: two OAPs on one host both with gRPCPort 18080\n# after: give each node a distinct port / bind each to its own interface\nadmin-server:\n  default:\n    gRPCPort: ${SW_ADMIN_SERVER_GRPC_PORT:18080}   # node A: 18080, node B: 18081","handlingStrategy":"retry","validationCode":"# Pre-flight: is the admin gRPC port free on this host?\nif ss -ltn 2>/dev/null | grep -q \":${SW_ADMIN_SERVER_GRPC_PORT} \"; then\n  echo \"admin gRPC port ${SW_ADMIN_SERVER_GRPC_PORT} already in use\"; exit 1;\nfi","typeGuard":null,"tryCatchPattern":"Catch ModuleStartException in orchestration/wrapper scripts (not inside OAP), inspect the cause for bind errors, free or reassign the port, then retry the start — retrying without changing anything will fail identically.","preventionTips":["Assign unique host ports per OAP node when using hostNetwork, and keep a port map for 11800/12800/admin HTTP/admin gRPC","In Kubernetes, use properly terminated pods (preStop/terminationGracePeriod) so restarted OAPs don't race their predecessor's sockets"],"tags":["admin-server","grpc","port-in-use","startup","deployment"],"backgroundTag":null,"analyzedSha":"102af09b4a56064e22050dded10e2c52e490d040","analyzedAt":"2026-08-14T10:47:52.647Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}