What causes "Upstream closed the subscriber channel." in XTLS/Xray-core?

Treat this as a normal stream-end signal: close the client stream and resubscribe if the instance is still alive

Upstream closed the subscriber channel.

Error message

Upstream closed the subscriber channel.

What it means

During a SubscribeRoutingStats stream, the subscriber channel was closed by the upstream stats channel (typically on shutdown or when the channel is stopped). The stream terminates with this error because no further values can arrive.

Source

Thrown at app/router/command/command.go:121

	}
	return AsProtobufMessage(request.FieldSelectors)(route), nil
}

func (s *routingServer) SubscribeRoutingStats(request *SubscribeRoutingStatsRequest, stream RoutingService_SubscribeRoutingStatsServer) error {
	if s.routingStats == nil {
		return errors.New("Routing statistics not enabled.")
	}
	genMessage := AsProtobufMessage(request.FieldSelectors)
	subscriber, err := stats.SubscribeRunnableChannel(s.routingStats)
	if err != nil {
		return err
	}
	defer stats.UnsubscribeClosableChannel(s.routingStats, subscriber)
	for {
		select {
		case value, ok := <-subscriber:
			if !ok {
				return errors.New("Upstream closed the subscriber channel.")
			}
			route, ok := value.(routing.Route)
			if !ok {
				return errors.New("Upstream sent malformed statistics.")
			}
			err := stream.Send(genMessage(route))
			if err != nil {
				return err
			}
		case <-stream.Context().Done():
			return stream.Context().Err()
		}
	}
}

func (s *routingServer) mustEmbedUnimplementedRoutingServiceServer() {}

type service struct {

View on GitHub (pinned to 7d214f8b09)

Solutions

Treat this as a normal stream-end signal: close the client stream and resubscribe if the instance is still alive
Keep subscriptions tied to the instance lifecycle (cancel before Close)
Retry with backoff in monitoring tooling that must survive restarts

Defensive patterns

Strategy: retry

Try / catch

for {
    err := subscribeRoutingStats(ctx)
    if err == nil || ctx.Err() != nil { break }
    if strings.Contains(err.Error(), "Upstream closed") {
        time.Sleep(backoff.Next()) // instance restarting; resubscribe
        continue
    }
    return err
}

Prevention

Tie subscription lifetime to the core instance; cancel streams before Close
Use reconnect-with-backoff in monitoring clients
Expect this error on graceful shutdown rather than treating it as a bug

When it happens

Trigger: xray instance shutting down or the stats channel being closed/stopped while a SubscribeRoutingStats client is still streaming.

Common situations: Long-lived subscribers across config reloads or process restarts; observers kept open after the core instance is closed.

Related errors

AI-assisted analysis of XTLS/Xray-core@7d214f8b09 (2026-08-15). Data as JSON: /api/errors/7ca39656c9343e46. Report an issue: GitHub.