vitessio/vitess · warning

listener closed: %w

Error message

listener closed: %w

What it means

vtadmin muxes gRPC and HTTP traffic over one port using a connection multiplexer (lmux); this error wraps the error returned when `lmux.Serve()` exits. Because the muxer feeds both the gRPC and HTTP servers, its failure or closure triggers a full shutdown via the shutdown channel. Typically this is observed when vtadmin is shutting down and the muxed listener is closed.

Source

Thrown at go/vt/vtadmin/grpcserver/server.go:253

	// Start the servers
	go func() {
		err := s.gRPCServer.Serve(grpcLis)
		err = fmt.Errorf("grpc server stopped: %w", err)
		log.Warn(fmt.Sprint(err))
		shutdown <- err
	}()

	go func() {
		err := http.Serve(anyLis, s.router)
		err = fmt.Errorf("http server stopped: %w", err)
		log.Warn(fmt.Sprint(err))
		shutdown <- err
	}()

	// Start muxing connections
	go func() {
		err := lmux.Serve()
		err = fmt.Errorf("listener closed: %w", err)
		log.Warn(fmt.Sprint(err))
		shutdown <- err
	}()

	for service := range s.gRPCServer.GetServiceInfo() {
		s.healthServer.SetServingStatus(service, healthpb.HealthCheckResponse_SERVING)
	}

	s.setServing(true)
	log.Info(fmt.Sprintf("server %s listening on %s", s.name, s.opts.Addr))

	reason := <-shutdown
	log.Warn(fmt.Sprintf("graceful shutdown triggered by: %v", reason))

	if s.opts.LameDuckDuration > 0 {
		log.Info(fmt.Sprintf("entering lame duck period for %v", s.opts.LameDuckDuration))
		s.healthServer.Shutdown()
		time.Sleep(s.opts.LameDuckDuration)

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Read the wrapped error to see whether the listener was closed deliberately or errored.
2. During shutdown, ignore as expected teardown noise.
3. Ensure ordering: start lmux.Serve before handing listeners to gRPC/HTTP and close via the single Close path.
4. If it occurs at startup, verify the configured port is free and bindable.

Example fix

// before
shutdown <- err
// after
if err != nil && !errors.Is(err, net.ErrClosed) {
	log.Warn(fmt.Sprintf("listener closed: %v", err))
}
shutdown <- err

if ln, err := net.Listen("tcp", addr); err != nil {
	log.Fatalf("cannot bind %s: %v", addr, err)
} else {
	ln.Close()
}

func isListenerClosed(err error) bool {
	return errors.Is(err, net.ErrClosed)
}

err := <-shutdownCh
if errors.Is(err, net.ErrClosed) {
	log.Info("muxed listener closed during shutdown")
} else {
	log.Error(fmt.Sprintf("lmux failed: %v", err))
}

When it happens

Trigger: `lmux.Serve()` returns: the underlying listener was closed (e.g., by Close()), an accept loop failed, or the muxer was already closed.

Common situations: Graceful vtadmin shutdown ordering (Close closes listener -> lmux.Serve returns); listener creation failed upstream; network accept errors on the serving port.

Related errors

• grpc server stopped: %w
• http server stopped: %w
• not allowed: deny-all security-policy enforced
• not allowed: read-only security-policy enforced
• invalid joined path

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/528d19340badcc8d. Report an issue: GitHub.