grpc/grpc-go · error

keepalive ping not acked within timeout %s

Error message

keepalive ping not acked within timeout %s

What it means

This error occurs in the server's keepalive goroutine when a keepalive ping was sent to the client but no acknowledgment (PING ACK) or any data was received within the configured keepalive timeout period. The server interprets this as a dead connection and closes it.

Source

Thrown at internal/transport/http2_server.go:1257

					t.logger.Infof("Closing server transport due to maximum connection age")
				}
				t.controlBuf.put(closeConnection{})
			case <-t.done:
			}
			return
		case <-kpTimer.C:
			lastRead := atomic.LoadInt64(&t.lastRead)
			if lastRead > prevNano {
				// There has been read activity since the last time we were
				// here. Setup the timer to fire at kp.Time seconds from
				// lastRead time and continue.
				outstandingPing = false
				kpTimer.Reset(time.Duration(lastRead) + t.kp.Time - time.Duration(time.Now().UnixNano()))
				prevNano = lastRead
				continue
			}
			if outstandingPing && kpTimeoutLeft <= 0 {
				t.Close(fmt.Errorf("keepalive ping not acked within timeout %s", t.kp.Timeout))
				return
			}
			if !outstandingPing {
				if channelz.IsOn() {
					t.channelz.SocketMetrics.KeepAlivesSent.Add(1)
				}
				t.controlBuf.put(p)
				kpTimeoutLeft = t.kp.Timeout
				outstandingPing = true
			}
			// The amount of time to sleep here is the minimum of kp.Time and
			// timeoutLeft. This will ensure that we wait only for kp.Time
			// before sending out the next ping (for cases where the ping is
			// acked).
			sleepDuration := min(t.kp.Time, kpTimeoutLeft)
			kpTimeoutLeft -= sleepDuration
			kpTimer.Reset(sleepDuration)
		case <-t.done:

View on GitHub (pinned to 03255a9237)

Solutions

1. Verify client health — check if the client process is alive and responsive.
2. Investigate network connectivity between client and server (firewall, NAT timeout, routing).
3. If the connection crosses a NAT/firewall, ensure the keepalive interval (Time) is shorter than the NAT idle timeout to prevent silent drops.
4. Tune keepalive.ServerParameters (Time and Timeout) to match your network's expected latency and reliability characteristics.

Example fix

// before (broken): default keepalive may not suit your network
srv := grpc.NewServer()

// after (valid): explicit keepalive tuned for the deployment
srv := grpc.NewServer(
  grpc.KeepaliveParams(keepalive.ServerParameters{
    Time:    10 * time.Second, // ping every 10s of inactivity
    Timeout: 5 * time.Second,  // wait 5s for ack before closing
  }),
)

// On the client side, a dead connection results in Unavailable status.
// gRPC retries automatically for idempotent RPCs. For others:
if status.Code(err) == codes.Unavailable {
    // connection was closed by server keepalive timeout
    // retry the RPC; gRPC will reconnect
    err = retryRPC(ctx, client, method, req)
}

When it happens

Trigger: The server sends a keepalive PING frame (after the keepalive Time interval with no read activity) and the client fails to respond with a PING ACK within keepalive.Timeout seconds. The server calls t.Close with this error, terminating all active streams on the connection.

Common situations: Client process crashed/hung without closing the connection (no FIN sent), network partition between client and server, client is alive but its network stack is blocked, very slow client that stops reading/writing, or asymmetric routing where the return path is broken.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• transport: timeout string is too short: %q
• transport: timeout string is too long: %q
• transport: timeout unit is not recognized: %q
• failed to write the HTTP request: %v
• received %d-bytes data exceeding the limit %d bytes

AI-assisted analysis of grpc/grpc-go@03255a9237 (2026-08-07). Data as JSON: /api/errors/1a0e5b1ae01050a7. Report an issue: GitHub.