grpc/grpc-go · error

keepalive ping not acked within timeout

Error message

keepalive ping not acked within timeout %s

What it means

Fires in http2Server.keepalive (http2_server.go:1257) when a keepalive PING sent to the peer was not acknowledged within the configured keepalive.Timeout. gRPC servers periodically PING idle connections to detect dead peers; if the peer never responds (or the network silently drops the ACK), the server closes the transport with this error to free resources.

Solutions

1. Increase keepalive.ServerParameters.Timeout to comfortably exceed observed network RTT and the time clients may stall before replying.
2. Ensure the client also enables keepalive (keepalive.ClientParameters{Time, Timeout, PermitWithoutStream}) so it responds to and sends PINGs.
3. Remove intermediaries that swallow HTTP/2 PINGs (some proxies/LBs do); use ones that forward PING frames.
4. Investigate why the peer isn't ACKing: check client liveness, GC pauses, thread starvation, or network blackholes.

Example fix

// before
srv := grpc.NewServer()
// default keepalive: server never pings, so dead peers linger

// after
srv := grpc.NewServer(
    grpc.KeepaliveParams(keepalive.ServerParameters{
        Time:    30 * time.Second, // ping after 30s idle
        Timeout: 10 * time.Second, // wait 10s for ACK (raise on slow networks)
    }),
)

// Set a server keepalive policy with a timeout that exceeds network RTT.
import "google.golang.org/grpc/keepalive"

srv := grpc.NewServer(grpc.KeepaliveParams(keepalive.ServerParameters{
    Time:    30 * time.Second,
    Timeout: 20 * time.Second,
}))

// The transport closes on keepalive failure; client RPCs get UNAVAILABLE.
// Retry with backoff on the client side.
if status.Code(err) == codes.Unavailable {
    // reconnect / retry per your retry policy
}

When it happens

Trigger: A server-side keepalive policy (keepalive.ServerParameters{Time, Timeout}) is active; after Time with no read activity the server sends a PING, and no ACK arrives before Timeout elapses. Typical when the client process is hung/dead-but-TCP-open, a NAT/firewall silently dropped the connection, or the network blackholes PING ACKs.

Common situations: Long-lived streaming RPCs where the client stalled; mobile clients that backgrounded and stopped responding; cloud LB/NAT idle reaping that half-opens the connection; clients behind a misconfigured proxy that doesn't forward PINGs; keepalive Timeout set too low for the network RTT.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• ErrCodeEnhanceYourCalm
• failed to do connect handshake, response
• failed to do connect handshake, status code
• failed to write the HTTP request
• gRPC requires HTTP/2

AI-assisted analysis of grpc/grpc-go@0c51461d27 (2026-08-11). Data as JSON: /api/errors/1a0e5b1ae01050a7. Report an issue: GitHub.

Appendix: source

Thrown at internal/transport/http2_server.go:1257

					t.logger.Infof("Closing server transport due to maximum connection age")
				}
				t.controlBuf.put(closeConnection{})
			case <-t.done:
			}
			return
		case <-kpTimer.C:
			lastRead := atomic.LoadInt64(&t.lastRead)
			if lastRead > prevNano {
				// There has been read activity since the last time we were
				// here. Setup the timer to fire at kp.Time seconds from
				// lastRead time and continue.
				outstandingPing = false
				kpTimer.Reset(time.Duration(lastRead) + t.kp.Time - time.Duration(time.Now().UnixNano()))
				prevNano = lastRead
				continue
			}
			if outstandingPing && kpTimeoutLeft <= 0 {
				t.Close(fmt.Errorf("keepalive ping not acked within timeout %s", t.kp.Timeout))
				return
			}
			if !outstandingPing {
				if channelz.IsOn() {
					t.channelz.SocketMetrics.KeepAlivesSent.Add(1)
				}
				t.controlBuf.put(p)
				kpTimeoutLeft = t.kp.Timeout
				outstandingPing = true
			}
			// The amount of time to sleep here is the minimum of kp.Time and
			// timeoutLeft. This will ensure that we wait only for kp.Time
			// before sending out the next ping (for cases where the ping is
			// acked).
			sleepDuration := min(t.kp.Time, kpTimeoutLeft)
			kpTimeoutLeft -= sleepDuration
			kpTimer.Reset(sleepDuration)
		case <-t.done:

View on GitHub (pinned to 0c51461d27)