{"record":{"id":"cc71af32584f385d","repo":"tailscale/tailscale","slug":"cannot-accept-connection-rate-limited","errorCode":null,"errorMessage":"cannot accept connection; rate limited","messagePattern":"cannot accept connection; rate limited","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"cmd/derper/derper.go","lineNumber":511,"sourceCode":"\tnumRejects expvar.Int\n\n\tnet.Listener\n\n\tlim *rate.Limiter\n}\n\nfunc newRateLimitedListener(ln net.Listener, limit rate.Limit, burst int) *rateLimitedListener {\n\treturn &rateLimitedListener{Listener: ln, lim: rate.NewLimiter(limit, burst)}\n}\n\nfunc (ln *rateLimitedListener) ExpVar() expvar.Var {\n\tm := new(metrics.Set)\n\tm.Set(\"counter_accepted_connections\", &ln.numAccepts)\n\tm.Set(\"counter_rejected_connections\", &ln.numRejects)\n\treturn m\n}\n\nvar errLimitedConn = errors.New(\"cannot accept connection; rate limited\")\n\nfunc (ln *rateLimitedListener) Accept() (net.Conn, error) {\n\t// Even under a rate limited situation, we accept the connection immediately\n\t// and close it, rather than being slow at accepting new connections.\n\t// This provides two benefits: 1) it signals to the client that something\n\t// is going on on the server, and 2) it prevents new connections from\n\t// piling up and occupying resources in the OS kernel.\n\t// The client will retry as needing (with backoffs in place).\n\tcn, err := ln.Listener.Accept()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif !ln.lim.Allow() {\n\t\tln.numRejects.Add(1)\n\t\tcn.Close()\n\t\treturn nil, errLimitedConn\n\t}\n\tln.numAccepts.Add(1)","sourceCodeStart":493,"sourceCodeEnd":529,"githubUrl":"https://github.com/tailscale/tailscale/blob/cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042/cmd/derper/derper.go#L493-L529","documentation":"derper wraps its listener with a token-bucket rate limiter (--accept-connection-limit rate with --accept-connection-burst burst; both effectively unlimited by default, derper.go:88-89). When the bucket is empty, Accept() returns errLimitedConn and bumps counter_rejected_connections; the comment notes connections are still accepted and immediately closed so clients see an active server and retry with backoff rather than piling up in the kernel backlog.","triggerScenarios":"derper started with a finite --accept-connection-limit (or -connection-rate-limit style config) and the incoming connection rate exceeding the rate+burst budget — e.g. after a derper restart when all clients reconnect at once, an aggressive health checker, or a connection flood.","commonSituations":"Operators who set a low accept rate for protection then see reconnect storms (mass client reconnect, LB health probes from many sources) hit the limit; visible via the derper expvar metrics counter_rejected_connections on /debug/vars.","solutions":["Treat it as expected behavior first — Tailscale clients retry with backoff, so brief bursts are harmless","If legitimate traffic is being shed, raise --accept-connection-limit and --accept-connection-burst to fit your client population","Check /debug/vars counter_rejected_connections over time to size the limit instead of guessing","If caused by health-check probes, spread or slow the probes rather than raising the global limit"],"exampleFix":"# before\nderper -a :443 --accept-connection-limit=5\n\n# after (sized for reconnect storms)\nderper -a :443 --accept-connection-limit=100 --accept-connection-burst=200","handlingStrategy":"retry","validationCode":"curl -s \"http://derper-host/debug/vars\" | python3 -c \"import json,sys; d=json.load(sys.stdin); r=d.get('counter_rejected_connections',{}); print('rejected:', r)\"","typeGuard":null,"tryCatchPattern":"// Client-side accept loop pattern when embedding derper's listener type:\nfor {\n    conn, err := ln.Accept()\n    if err != nil {\n        if err.Error() == \"cannot accept connection; rate limited\" || errors.Is(err, errLimitedConn) {\n            time.Sleep(backoff.Next()) // server closed conn intentionally; retry with backoff\n            continue\n        }\n        return err // real listener error\n    }\n    go handle(conn)\n}","preventionTips":["Monitor counter_rejected_connections vs counter_accepted_connections on /debug/vars to size limits from data","Account for reconnect storms when sizing --accept-connection-limit (all clients reconnect after a derper restart)","Keep burst >= expected simultaneous reconnect wave to shed only true floods"],"tags":["derper","rate-limiting","network","operations"],"backgroundTag":null,"analyzedSha":"cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042","analyzedAt":"2026-08-15T19:58:31.583Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}