OpenNHP/opennhp · error

NHP Server timeout

Error message

NHP Server timeout

What it means

After the relay successfully enqueues the NHP_RLY forward to the NHP server, it waits on a response channel for the raw encrypted ACK/COK packet. If no response arrives within udpTimeout milliseconds (rs.config.UDPTimeoutMs, default defaultUDPTimeoutMs), handleRelay returns HTTP 504 Gateway Timeout with body 'NHP Server timeout'. This mirrors an upstream-timeout semantic: the relay accepted the request but the NHP server never answered in time.

Solutions

1. Verify the NHP server is healthy and responding to knocks directly (bypass the relay) to isolate where the delay is.
2. Increase UDPTimeoutMs in the relay's config.toml if legitimate server latency exceeds the current timeout.
3. Check network path relay->server for UDP loss/MTU issues; NHP runs over encrypted UDP which some middleboxes drop.
4. Check relay logs for 'ambiguity check' or counter-dispatch warnings indicating the response arrived but was not routed to this waiter.
5. Implement client retry on 504, ideally with a fresh inner counter, since the original transaction may have completed server-side.

Example fix

// before: single attempt, hard failure on 504
resp, err := http.Post(relayURL, "application/octet-stream", body)
if resp.StatusCode == http.StatusGatewayTimeout {
    return fmt.Errorf("NHP Server timeout")
}

// after: retry with fresh transaction on timeout
for attempt := 0; attempt < 2; attempt++ {
    body = buildKnockWithFreshCounter()
    resp, err := http.Post(relayURL, "application/octet-stream", body)
    if err == nil && resp.StatusCode == http.StatusOK {
        return nil
    }
}
return fmt.Errorf("NHP server did not respond in time after retries")

// probe server responsiveness before sending real knocks
func serverResponds(relayURL string, timeout time.Duration) bool {
    client := http.Client{Timeout: timeout}
    resp, err := client.Post(relayURL, "application/octet-stream", probePayload())
    if err != nil { return false }
    resp.Body.Close()
    return resp.StatusCode == http.StatusOK
}

func isNHPTimeout(err error) bool {
    var httpErr *HTTPStatusError
    if errors.As(err, &httpErr) {
        return httpErr.StatusCode == http.StatusGatewayTimeout &&
            strings.Contains(httpErr.Body, "NHP Server timeout")
    }
    return false
}

resp, err := client.Do(req)
if err != nil { return err }
if resp.StatusCode == http.StatusGatewayTimeout {
    // treat as inconclusive: the server may still have processed the knock
    if isRetryable(payload) {
        return retryWithFreshCounter(payload)
    }
    return ErrUpstreamTimeout
}

When it happens

Trigger: A forward was sent (rs.sendMsgCh accepted the packet) but the server's ACK/COK never reached connectionRoutine within udpTimeout: (1) UDP packet loss between relay and server, (2) the NHP server is slow or hung, (3) server responds with a counter that fails the pendingRequests ambiguity check so the response is never dispatched to responseCh, (4) UDPTimeoutMs configured too small for the server's real latency.

Common situations: Deploying the relay in a region with poor connectivity to the NHP server; server-side processing slowdown under load (e.g. AC firewall operations taking longer than the timeout); clock/latency sensitivity after lowering UDPTimeoutMs for faster failure detection; duplicate inner counters from a misbehaving client causing responses to be dropped by the ambiguity check.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• relay overloaded
• could not send https request
• relay source address
• inner packet too large
• rkn rate limit exceeded

AI-assisted analysis of OpenNHP/opennhp@6e04ca5ff0 (2026-09-07). Data as JSON: /api/errors/bed2fe761856ccf8. Report an issue: GitHub.

Appendix: source

Thrown at endpoints/relay/relay.go:1127

			udpTimeout, innerCounter, realAddr, cr.id)
		http.Error(w, "relay overloaded", http.StatusServiceUnavailable)
		return
	}

	// Wait for the raw encrypted ACK/COK packet from the server.
	select {
	case rawBytes := <-responseCh:
		log.Info("[Relay] received response for inner counter=%d, %d raw bytes, forwarding to client %s (server %s)",
			innerCounter, len(rawBytes), realAddr, cr.id)

		w.Header().Set("Content-Type", "application/octet-stream")
		w.WriteHeader(http.StatusOK)
		_, _ = w.Write(rawBytes)

	case <-time.After(time.Duration(udpTimeout) * time.Millisecond):
		log.Warning("[Relay] timeout waiting for server response (inner counter=%d, client %s, server %s)",
			innerCounter, realAddr, cr.id)
		http.Error(w, "NHP Server timeout", http.StatusGatewayTimeout)
	}
}

// realClientAddr returns the originating address of an HTTP request as a
// *net.UDPAddr so it can be encoded in the RelayForwardMsg.
//
// When the direct TCP peer is on the loopback interface — i.e. a local
// reverse proxy (nginx, etc.) forwarded the request — the proxy's view
// of the real client is taken from X-Real-IP, which the proxy is
// expected to overwrite unconditionally (e.g. nginx
// `proxy_set_header X-Real-IP $remote_addr;`).
//
// X-Forwarded-For is intentionally NOT consulted: nginx's standard
// `$proxy_add_x_forwarded_for` *appends* to whatever XFF the client
// sent, so its first entry is attacker-controlled. Trusting XFF would
// let any HTTP client choose the SourceAddr that flows to nhp-server
// and ultimately to the AC ipset rule, defeating the per-source-IP
// authorization model.

View on GitHub (pinned to 6e04ca5ff0)