vitessio/vitess · warning

grpc server stopped: %w

Error message

grpc server stopped: %w

What it means

This error is produced by vtadmin's gRPC server goroutine when `grpcServer.Serve()` returns, which only happens if the server stops unexpectedly (it blocks until failure or explicit Stop). The returned error is wrapped with 'grpc server stopped' and sent to the shutdown channel, which triggers the whole vtadmin server shutdown. It is a signal that the gRPC listener died, not necessarily a fatal bug — during graceful shutdown Serve returns a normal error.

Source

Thrown at go/vt/vtadmin/grpcserver/server.go:238

		sig := <-signals
		err := fmt.Errorf("received signal: %v", sig)
		log.Warn(fmt.Sprint(err))
		shutdown <- err
	}()

	if s.opts.MetricsEndpoint != "" {
		if !strings.HasPrefix(s.opts.MetricsEndpoint, "/") {
			s.opts.MetricsEndpoint = "/" + s.opts.MetricsEndpoint
		}

		grpc_prometheus.Register(s.gRPCServer)
		s.router.Handle(s.opts.MetricsEndpoint, promhttp.Handler())
	}

	// Start the servers
	go func() {
		err := s.gRPCServer.Serve(grpcLis)
		err = fmt.Errorf("grpc server stopped: %w", err)
		log.Warn(fmt.Sprint(err))
		shutdown <- err
	}()

	go func() {
		err := http.Serve(anyLis, s.router)
		err = fmt.Errorf("http server stopped: %w", err)
		log.Warn(fmt.Sprint(err))
		shutdown <- err
	}()

	// Start muxing connections
	go func() {
		err := lmux.Serve()
		err = fmt.Errorf("listener closed: %w", err)
		log.Warn(fmt.Sprint(err))
		shutdown <- err
	}()

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Check the wrapped inner error in the log to find the root cause (listener closed vs. Serve called twice).
2. If seen at shutdown, ignore it — it is the expected path when the process is terminating.
3. Ensure nothing in your setup closes the gRPC listener independently of grpcserver.Close().
4. Verify Serve() is only invoked once per grpcserver.Server instance.

Example fix

// before
go func() {
	if err := srv.Serve(lis); err != nil {
		log.Fatal(err)
	}
}()
// after
go func() {
	err := srv.Serve(lis)
	if err != nil && !errors.Is(err, grpc.ErrServerStopped) {
		log.Warn(fmt.Sprintf("grpc server stopped: %v", err))
	}
	shutdown <- err
}()

func isExpectedStop(err error) bool {
	return err == nil || errors.Is(err, grpc.ErrServerStopped) || errors.Is(err, net.ErrClosed)
}

err := <-shutdownCh
if isExpectedStop(err) {
	log.Info("vtadmin shutting down")
} else {
	log.Error(fmt.Sprintf("unexpected server stop: %v", err))
}

When it happens

Trigger: Calling grpcserver.Server.Serve() and the underlying gRPC server stops: the listener is closed by another component, Serve is called twice, the listener fails to accept connections, or the server is being shut down via Close/GracefulStop.

Common situations: Another goroutine closes the shared listener while vtadmin is running; port hijacked or listener error at OS level; vtadmin shutting down normally and this warning appearing in logs as part of the shutdown sequence.

Related errors

• GetKeyspaces(cluster = %s): %w
• received signal: %v
• http server stopped: %w
• listener closed: %w
• error dialing vtgate: %w

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/fe86afb67e536086. Report an issue: GitHub.