lrs: failed to receive first LoadStatsResponse: %v

Error message

lrs: failed to receive first LoadStatsResponse: %v

What it means

Thrown by recvFirstLoadStatsResponse in the LRS client when stream.Recv() returns a non-nil error before the first LoadStatsResponse arrives. The gRPC LRS runner catches it (logs 'Reading from LRS streaming RPC failed'), returns nil, and backoff.RunF retries the whole stream with exponential backoff, so it is transient by design. It surfaces to a developer only if they instrument the stream directly or if retries keep failing.

Source

Thrown at internal/xds/clients/lrsclient/lrs_stream.go:207

		return err
	}
	err = stream.Send(msg)
	if err == io.EOF {
		return getStreamError(stream)
	}
	return err
}

// recvFirstLoadStatsResponse receives the first LoadStatsResponse from the LRS
// server.  Returns the following:
//   - a list of cluster names requested by the server or an empty slice if the
//     server requested for load from all clusters
//   - the load reporting interval, and
//   - any error encountered
func (lrs *streamImpl) recvFirstLoadStatsResponse(stream clients.Stream) ([]string, time.Duration, error) {
	r, err := stream.Recv()
	if err != nil {
		return nil, 0, fmt.Errorf("lrs: failed to receive first LoadStatsResponse: %v", err)
	}
	var resp v3lrspb.LoadStatsResponse
	if err := proto.Unmarshal(r, &resp); err != nil {
		if lrs.logger.V(2) {
			lrs.logger.Infof("Failed to unmarshal response to LoadStatsResponse: %v", err)
		}
		return nil, time.Duration(0), fmt.Errorf("lrs: unexpected message type %T", r)
	}
	if lrs.logger.V(perRPCVerbosityLevel) {
		lrs.logger.Infof("Received first LoadStatsResponse: %s", pretty.ToJSON(&resp))
	}

	internal := resp.GetLoadReportingInterval()
	if internal.CheckValid() != nil {
		return nil, 0, fmt.Errorf("lrs: invalid load_reporting_interval: %v", err)
	}
	loadReportingInterval := internal.AsDuration()

View on GitHub (pinned to 03255a9237)

Solutions

Verify the ServerURI and port in the clients.ServerIdentifier passed to ReportLoad resolve and are reachable (e.g. grpcurl to the LRS service path).
Check that the TransportBuilder supplies credentials the LRS server accepts (mTLS/insecure as appropriate) and that the server speaks the v3 LRS service.
Inspect the wrapped %v cause: io.EOF/Unavailable points to connection/TLS, PermissionDenied/Unauthenticated to credentials, Unimplemented to a non-LRS server.
Confirm the management server is actually configured to serve load reports for the requested clusters; a server that never responds keeps the stream open without error, so a Recv error usually means the connection itself broke.

Example fix

// before: server URI typo / wrong port
si := clients.ServerIdentifier{ServerURI: "trafficdirector.googleapis.com:443"}
ls, err := lrsClient.ReportLoad(si)

// after: correct LRS endpoint + valid authority
si := clients.ServerIdentifier{
  ServerURI: "trafficdirector.googleapis.com:443",
  Authority: "td-token", // matches bootstrap credentials
}

Defensive patterns

Strategy: retry

Validate before calling

// Pre-check reachability before opening the LRS stream.
func lrsReachable(ctx context.Context, si clients.ServerIdentifier) error {
    host, port, err := net.SplitHostPort(si.ServerURI)
    if err != nil { return err }
    d := net.Dialer{Timeout: 3 * time.Second}
    conn, err := d.DialContext(ctx, "tcp", net.JoinHostPort(host, port))
    if err != nil { return err }
    conn.Close()
    return nil
}

Try / catch

// The LRS client already retries internally; treat ReportLoad errors as
// transient and back off yourself if you drive reconnection.
ls, err := client.ReportLoad(si)
if err != nil {
    logger.Warnf("LRS unavailable, will retry: %v", err)
    return
}

Prevention

Validate ServerIdentifier URI and credentials in the TransportBuilder before calling ReportLoad.
Treat first-response errors as transient; the runner applies exponential backoff automatically.
Monitor for sustained failure rather than a single occurrence.

When it happens

Trigger: Calling lrsclient.ReportLoad against an LRS server that is unreachable, refuses the StreamLoadStats RPC, drops the connection, fails a TLS handshake, or never sends a first response (idle server). The error is the raw gRPC stream error from Recv on '/envoy.service.load_stats.v3.LoadReportingService/StreamLoadStats'.

Common situations: Wrong ServerURI or port in the LRS server identifier, missing/mismatched TLS credentials in the transport builder, LRS management server not actually implementing the LRS service, network partition or firewall between client and server, or the server closing the stream immediately (e.g. node rejected).

Related errors

AI-assisted analysis of grpc/grpc-go@03255a9237 (2026-08-07). Data as JSON: /api/errors/036e7efe17671af4. Report an issue: GitHub.