{"record":{"id":"1a0e5b1ae01050a7","repo":"grpc/grpc-go","slug":"keepalive-ping-not-acked-within-timeout-s","errorCode":null,"errorMessage":"keepalive ping not acked within timeout %s","messagePattern":"keepalive ping not acked within timeout (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/transport/http2_server.go","lineNumber":1257,"sourceCode":"\t\t\t\t\tt.logger.Infof(\"Closing server transport due to maximum connection age\")\n\t\t\t\t}\n\t\t\t\tt.controlBuf.put(closeConnection{})\n\t\t\tcase <-t.done:\n\t\t\t}\n\t\t\treturn\n\t\tcase <-kpTimer.C:\n\t\t\tlastRead := atomic.LoadInt64(&t.lastRead)\n\t\t\tif lastRead > prevNano {\n\t\t\t\t// There has been read activity since the last time we were\n\t\t\t\t// here. Setup the timer to fire at kp.Time seconds from\n\t\t\t\t// lastRead time and continue.\n\t\t\t\toutstandingPing = false\n\t\t\t\tkpTimer.Reset(time.Duration(lastRead) + t.kp.Time - time.Duration(time.Now().UnixNano()))\n\t\t\t\tprevNano = lastRead\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif outstandingPing && kpTimeoutLeft <= 0 {\n\t\t\t\tt.Close(fmt.Errorf(\"keepalive ping not acked within timeout %s\", t.kp.Timeout))\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !outstandingPing {\n\t\t\t\tif channelz.IsOn() {\n\t\t\t\t\tt.channelz.SocketMetrics.KeepAlivesSent.Add(1)\n\t\t\t\t}\n\t\t\t\tt.controlBuf.put(p)\n\t\t\t\tkpTimeoutLeft = t.kp.Timeout\n\t\t\t\toutstandingPing = true\n\t\t\t}\n\t\t\t// The amount of time to sleep here is the minimum of kp.Time and\n\t\t\t// timeoutLeft. This will ensure that we wait only for kp.Time\n\t\t\t// before sending out the next ping (for cases where the ping is\n\t\t\t// acked).\n\t\t\tsleepDuration := min(t.kp.Time, kpTimeoutLeft)\n\t\t\tkpTimeoutLeft -= sleepDuration\n\t\t\tkpTimer.Reset(sleepDuration)\n\t\tcase <-t.done:","sourceCodeStart":1239,"sourceCodeEnd":1275,"githubUrl":"https://github.com/grpc/grpc-go/blob/0c51461d27177d997e14c642fe18c11668fc09a3/internal/transport/http2_server.go#L1239-L1275","documentation":"Fires in http2Server.keepalive (http2_server.go:1257) when a keepalive PING sent to the peer was not acknowledged within the configured keepalive.Timeout. gRPC servers periodically PING idle connections to detect dead peers; if the peer never responds (or the network silently drops the ACK), the server closes the transport with this error to free resources.","triggerScenarios":"A server-side keepalive policy (keepalive.ServerParameters{Time, Timeout}) is active; after Time with no read activity the server sends a PING, and no ACK arrives before Timeout elapses. Typical when the client process is hung/dead-but-TCP-open, a NAT/firewall silently dropped the connection, or the network blackholes PING ACKs.","commonSituations":"Long-lived streaming RPCs where the client stalled; mobile clients that backgrounded and stopped responding; cloud LB/NAT idle reaping that half-opens the connection; clients behind a misconfigured proxy that doesn't forward PINGs; keepalive Timeout set too low for the network RTT.","solutions":["Increase keepalive.ServerParameters.Timeout to comfortably exceed observed network RTT and the time clients may stall before replying.","Ensure the client also enables keepalive (keepalive.ClientParameters{Time, Timeout, PermitWithoutStream}) so it responds to and sends PINGs.","Remove intermediaries that swallow HTTP/2 PINGs (some proxies/LBs do); use ones that forward PING frames.","Investigate why the peer isn't ACKing: check client liveness, GC pauses, thread starvation, or network blackholes."],"exampleFix":"// before\nsrv := grpc.NewServer()\n// default keepalive: server never pings, so dead peers linger\n\n// after\nsrv := grpc.NewServer(\n    grpc.KeepaliveParams(keepalive.ServerParameters{\n        Time:    30 * time.Second, // ping after 30s idle\n        Timeout: 10 * time.Second, // wait 10s for ACK (raise on slow networks)\n    }),\n)","handlingStrategy":"validation","validationCode":"// Set a server keepalive policy with a timeout that exceeds network RTT.\nimport \"google.golang.org/grpc/keepalive\"\n\nsrv := grpc.NewServer(grpc.KeepaliveParams(keepalive.ServerParameters{\n    Time:    30 * time.Second,\n    Timeout: 20 * time.Second,\n}))","typeGuard":null,"tryCatchPattern":"// The transport closes on keepalive failure; client RPCs get UNAVAILABLE.\n// Retry with backoff on the client side.\nif status.Code(err) == codes.Unavailable {\n    // reconnect / retry per your retry policy\n}","preventionTips":["Enable keepalive on both client and server with compatible Time/Timeout.","Set Timeout comfortably above expected RTT and GC pause durations.","Ensure proxies/LBs forward HTTP/2 PING frames."],"tags":["keepalive","transport","network","server","dead-peer"],"backgroundTag":null,"analyzedSha":"0c51461d27177d997e14c642fe18c11668fc09a3","analyzedAt":"2026-08-11T14:49:15.055Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}