apache/skywalking · critical · ModuleStartException

admin-server: failed to start gRPC server

Error message

admin-server: failed to start gRPC server

What it means

AdminServerModuleProvider.start() calls grpcServer.start() for the admin-internal gRPC bus; a ServerException (typically bind failure: port already in use, or missing permission to bind a privileged port) is wrapped in ModuleStartException. This happens after config validation, at actual socket bind time, and stops OAP startup.

Source

Thrown at oap-server/server-admin/admin-server/src/main/java/org/apache/skywalking/oap/server/admin/server/module/AdminServerModuleProvider.java:222

        // already registered in prepare() with a lazy ClusterNodesQuery
        // supplier; nothing else to do here.
    }

    @Override
    public void notifyAfterCompleted() throws ModuleStartException {
        if (RunningMode.isInitMode()) {
            return;
        }
        try {
            if (grpcServer != null) {
                grpcServer.start();
                log.info("admin-server gRPC listening on {}:{} (peer-to-peer admin RPCs only — "
                             + "MUST be reachable between OAP nodes; MUST NOT be exposed to the agent network "
                             + "or operators).",
                         moduleConfig.getGRPCHost(), moduleConfig.getGRPCPort());
            }
        } catch (final ServerException e) {
            throw new ModuleStartException("admin-server: failed to start gRPC server", e);
        }
        if (peerChannelManager != null) {
            peerChannelManager.start();
        }
        if (httpServer != null) {
            httpServer.start();
            log.info(
                "admin-server HTTP listening on {}:{} (no built-in authentication — "
                    + "gateway-protect with an IP allow-list and authenticating reverse proxy; "
                    + "never expose to the public internet).",
                moduleConfig.getHost(), moduleConfig.getPort()
            );
        }
    }

    @Override
    public String[] requiredModules() {
        return new String[] {

View on GitHub (pinned to 102af09b4a)

Solutions

1. Identify the holder of the port: ss -ltnp | grep <port> or netstat -ltnp
2. Free the port or move admin-server gRPCPort to an unused one, then restart
3. If binding a port < 1024, run with adequate capability or choose a high port
4. For flapping containers, ensure previous pods are fully terminated before replacements start

Example fix

# before: two OAPs on one host both with gRPCPort 18080
# after: give each node a distinct port / bind each to its own interface
admin-server:
  default:
    gRPCPort: ${SW_ADMIN_SERVER_GRPC_PORT:18080}   # node A: 18080, node B: 18081

# Pre-flight: is the admin gRPC port free on this host?
if ss -ltn 2>/dev/null | grep -q ":${SW_ADMIN_SERVER_GRPC_PORT} "; then
  echo "admin gRPC port ${SW_ADMIN_SERVER_GRPC_PORT} already in use"; exit 1;
fi

Catch ModuleStartException in orchestration/wrapper scripts (not inside OAP), inspect the cause for bind errors, free or reassign the port, then retry the start — retrying without changing anything will fail identically.

When it happens

Trigger: Another process (or a second OAP instance on the same host, or a misconfigured pod with two replicas sharing network) already listens on the configured gRPCPort; or the port is below 1024 and the process lacks CAP_NET_BIND_SERVICE.

Common situations: Running two OAP nodes with hostNetwork on the same machine; port collision between admin gRPCPort and another service; container restart where the old process still holds the port (TIME_WAIT/liveness overlap); Kubernetes NodePort/hostPort conflicts.

Related errors

• admin-server: gRPCPort must be > 0 when the module is enable
• admin-server: failed to build admin gRPC client SSL context
• admin-server: port must be > 0 when the module is enabled, g
• admin-server: gRPCSslEnabled=true but gRPCSslTrustedCAsPath
• No host setting.

AI-assisted analysis of apache/skywalking@102af09b4a (2026-08-14). Data as JSON: /api/errors/d75adc55c2019913. Report an issue: GitHub.