GoogleCloudPlatform/microservices-demo · critical

grpc: failed to connect %s

Error message

grpc: failed to connect %s

What it means

mustConnGRPC in checkoutservice creates a gRPC client connection via grpc.NewClient and panics with errors.Wrapf(err, "grpc: failed to connect %s", addr) if connection setup fails. Because NewClient is lazy, this error usually reflects an invalid target address (unparsable host/port, bad scheme) rather than a live-dial failure. Called from main and initTracing, a panic here means the checkout service cannot start.

Source

Thrown at src/checkoutservice/main.go:218

}

func mustMapEnv(target *string, envKey string) {
	v := os.Getenv(envKey)
	if v == "" {
		panic(fmt.Sprintf("environment variable %q not set", envKey))
	}
	*target = v
}

func mustConnGRPC(ctx context.Context, conn **grpc.ClientConn, addr string) {
	var err error
	_, cancel := context.WithTimeout(ctx, time.Second*3)
	defer cancel()
	*conn, err = grpc.NewClient(addr,
		grpc.WithTransportCredentials(insecure.NewCredentials()),
		grpc.WithStatsHandler(otelgrpc.NewClientHandler()))
	if err != nil {
		panic(errors.Wrapf(err, "grpc: failed to connect %s", addr))
	}
}

func (cs *checkoutService) Check(ctx context.Context, req *healthpb.HealthCheckRequest) (*healthpb.HealthCheckResponse, error) {
	return &healthpb.HealthCheckResponse{Status: healthpb.HealthCheckResponse_SERVING}, nil
}

func (cs *checkoutService) Watch(req *healthpb.HealthCheckRequest, ws healthpb.Health_WatchServer) error {
	return status.Errorf(codes.Unimplemented, "health check via Watch not implemented")
}

func (cs *checkoutService) PlaceOrder(ctx context.Context, req *pb.PlaceOrderRequest) (*pb.PlaceOrderResponse, error) {
	log.Infof("[PlaceOrder] user_id=%q user_currency=%q", req.UserId, req.UserCurrency)

	orderID, err := uuid.NewUUID()
	if err != nil {
		return nil, status.Errorf(codes.Internal, "failed to generate order uuid")
	}

View on GitHub (pinned to 72ba613a05)

Solutions

1. Verify the address env var (e.g. SHIPPING_SERVICE_ADDR / PAYMENT_SERVICE_ADDR) is set to host:port
2. Confirm the target service exists and DNS resolves from the checkout pod (nslookup)
3. Check kubernetes service name and namespace match the configured address
4. Ensure the port matches the gRPC container port, not the plain HTTP one
5. Test the address string format (scheme/host:port) accepted by grpc.NewClient

Example fix

// before
*conn, err = grpc.NewClient(addr, ...)
if err != nil {
    panic(errors.Wrapf(err, "grpc: failed to connect %s", addr))
}
// after
if addr == "" {
    panic("grpc: target address is empty — set the service addr env var")
}
*conn, err = grpc.NewClient(addr, ...)
if err != nil {
    panic(errors.Wrapf(err, "grpc: failed to create client for %s", addr))
}

if addr == "" {
    panic("grpc: service address env var is empty")
}
if _, _, err := net.SplitHostPort(addr); err != nil {
    panic(fmt.Sprintf("grpc: invalid target %q: %v", addr, err))
}

defer func() {
    if r := recover(); r != nil {
        log.Fatalf("checkout startup failed: %v", r)
    }
}()

When it happens

Trigger: grpc.NewClient(addr, insecure creds, otel stats handler) returns err: malformed/unresolvable service address (empty env var, bad DNS name, missing port), or unsupported scheme in the target string.

Common situations: Required *_SERVICE_ADDR env var not set or empty in the deployment manifest; Kubernetes service name typo; port omitted from the address; wrong DNS in a different namespace; migrating from grpc.Dial to grpc.NewClient with a target string format the resolver rejects.

Related errors

• PRODUCT_CATALOG_SERVICE_ADDR environment variable not set
• shipment failed: %+v
• InvalidCreditCard
• UnacceptedCreditCard
• ExpiredCreditCard

AI-assisted analysis of GoogleCloudPlatform/microservices-demo@72ba613a05 (2026-09-02). Data as JSON: /api/errors/8d080a5a7af8717e. Report an issue: GitHub.