What causes "failed to receive ALTS handshaker response: %w" in grpc/grpc-go?

Detect io.EOF / context errors via errors.Is on the wrapped error and tear down + re-create the handshaker rather than looping on the dead stream.

grpc/grpc-go · error

failed to receive ALTS handshaker response: %w

Error message

failed to receive ALTS handshaker response: %w

What it means

Returned by accessHandshakerService when h.stream.Recv() fails while waiting for the handshaker service's HandshakerResp. %w wraps the receive error. The stream is broken on the read side: the service closed it (io.EOF), sent a GOAWAY, the connection reset, or the context expired.

Source

Thrown at credentials/alts/internal/handshaker/handshaker.go:309

	}
	maxFrameSize := int(envconfig.ALTSMaxFrameSize)
	if peerMax := int(result.GetMaxFrameSize()); peerMax > 0 {
		maxFrameSize = min(peerMax, maxFrameSize)
	}
	sc, err := conn.NewConnWithMaxFrameSize(h.conn, h.side, result.GetRecordProtocol(), result.KeyData[:keyLen], extra, maxFrameSize)
	if err != nil {
		return nil, nil, err
	}
	return sc, result, nil
}

func (h *altsHandshaker) accessHandshakerService(req *altspb.HandshakerReq) (*altspb.HandshakerResp, error) {
	if err := h.stream.Send(req); err != nil {
		return nil, fmt.Errorf("failed to send ALTS handshaker request: %w", err)
	}
	resp, err := h.stream.Recv()
	if err != nil {
		return nil, fmt.Errorf("failed to receive ALTS handshaker response: %w", err)
	}
	return resp, nil
}

// processUntilDone processes the handshake until the handshaker service returns
// the results. Handshaker service takes care of frame parsing, so we read
// whatever received from the network and send it to the handshaker service.
func (h *altsHandshaker) processUntilDone(resp *altspb.HandshakerResp, extra []byte) (*altspb.HandshakerResult, []byte, error) {
	var lastWriteTime time.Time
	buf := make([]byte, frameLimit)
	for {
		if len(resp.OutFrames) > 0 {
			lastWriteTime = time.Now()
			if _, err := h.conn.Write(resp.OutFrames); err != nil {
				return nil, nil, err
			}
		}
		if resp.Result != nil {

View on GitHub (pinned to 03255a9237)

Solutions

Detect io.EOF / context errors via errors.Is on the wrapped error and tear down + re-create the handshaker rather than looping on the dead stream.
Ensure the ClientConn to the handshaker service reconnects automatically (default grpc behavior) and that handshakes are re-initiated on new connections.
Raise the handshake context timeout above the observed metadata-server round-trip latency.
Monitor the handshaker service health and metadata-server availability if this spikes.

Example fix

// before: retry on a dead stream
for {
    resp, err := h.accessHandshakerService(req)
    if err != nil { continue }
}

// after: tear down and rebuild on receive failure
resp, err := h.accessHandshakerService(req)
if err != nil {
    h.Close()
    return nil, err // caller re-runs ClientHandshake on a fresh handshaker
}

Defensive patterns

Strategy: retry

Validate before calling

// Pre-check that the stream is still open before relying on Recv.
func (h *altsHandshaker) streamAlive() bool { return h.stream != nil }

Try / catch

resp, err := h.accessHandshakerService(req)
if err != nil {
    if errors.Is(err, io.EOF) {
        // service closed the stream cleanly: rebuild handshaker.
        h.Close()
    }
    if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
        return nil, err
    }
    return nil, fmt.Errorf("handshaker recv failed: %w", err)
}

Prevention

Re-create the handshaker (Close + NewClientHandshaker) once Recv fails.
Distinguish io.EOF (clean close) from network resets in error handling.
Do not busy-loop on a dead stream.
Keep the metadata-server reachable and stable to avoid mid-handshake teardowns.

When it happens

Trigger: Calling Recv on the DoHandshake stream after the handshaker service terminated it — service-side error followed by close, network interruption mid-handshake, context cancellation, or the metadata server process restarting. Always fires together with or shortly after [163] once the stream is torn down.

Common situations: Metadata server restarts on GKE upgrades, network policy killing idle streams, handshaker service OOM/crash, or handshake loops that keep retrying on a dead stream instead of re-dialing.

Understand the failure class

SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

AI-assisted analysis of grpc/grpc-go@03255a9237 (2026-08-07). Data as JSON: /api/errors/e05e48a7ef5d4fee. Report an issue: GitHub.