{"record":{"id":"abd02c4ccc82eb95","repo":"OpenNHP/opennhp","slug":"server-has-no-usable-instance","errorCode":null,"errorMessage":"server has no usable instance","messagePattern":"server has no usable instance","errorType":"http","errorClass":null,"httpStatus":503,"severity":"error","filePath":"endpoints/relay/relay.go","lineNumber":1003,"sourceCode":"\trealAddr, err := realClientAddr(r)\n\tif err != nil {\n\t\tlog.Error(\"[Relay] %v\", err)\n\t\thttp.Error(w, \"relay misconfigured: missing X-Real-IP header from local reverse proxy\", http.StatusBadGateway)\n\t\treturn\n\t}\n\trealAddrKey := realAddr.String()\n\n\t// Pick a target instance. When StickyInstance is enabled,\n\t// hash the real client IP so the same client always reaches the same\n\t// instance — required for stateful flows like OTP→REG where per-\n\t// instance local state (SQLite) must be consistent across requests.\n\t// When disabled (default), each request is load-balanced independently.\n\tvar inst *serverInstance\n\tif cr.sticky && len(cr.instances) > 1 {\n\t\tvar ok bool\n\t\tinst, ok = cr.picker.PickByKey(realAddrKey)\n\t\tif !ok {\n\t\t\thttp.Error(w, \"server has no usable instance\", http.StatusServiceUnavailable)\n\t\t\treturn\n\t\t}\n\t} else {\n\t\tinst = cr.pickInstance()\n\t}\n\tif inst == nil {\n\t\thttp.Error(w, \"server has no usable instance\", http.StatusServiceUnavailable)\n\t\treturn\n\t}\n\n\t// Bound concurrent forwards (and thus pendingRequests size) per\n\t// instance. Non-blocking acquire: if the instance is saturated, shed\n\t// the request with 503 rather than queueing — the alternative is the\n\t// pending map growing unbounded when an adversary opens faster than\n\t// the 5s handler timeout drains. Released on handler return.\n\tselect {\n\tcase inst.inFlight <- struct{}{}:\n\t\tdefer func() { <-inst.inFlight }()","sourceCodeStart":985,"sourceCodeEnd":1021,"githubUrl":"https://github.com/OpenNHP/opennhp/blob/6e04ca5ff03222a699c24205cd4bf8fee9af7ffe/endpoints/relay/relay.go#L985-L1021","documentation":"handleRelay responds with HTTP 503 'server has no usable instance' when sticky sessions are enabled, multiple instances exist, and the picker cannot obtain an instance for the client's hashed address key (PickByKey returned !ok). The relay cannot forward the packet because no backend instance is available for this client's sticky slot.","triggerScenarios":"POSTing to a relay whose sticky option is on with >1 configured instances while the sticky picker fails to resolve a key — e.g. no healthy instances registered, picker initialization failure, or all instances for that key removed.","commonSituations":"Backend NHP servers down so instances were pruned from the picker; misconfigured server table leaving zero healthy instances; race during shutdown where instances were drained before requests stopped arriving.","solutions":["Verify at least one backend NHP server instance is running and reachable from the relay","Check the relay's server configuration table for correct addresses and reload it","Inspect instance health/remove logic logs to see why the picker has no entry for the key","Temporarily disable StickyInstance (single-instance or LB mode) to confirm the backend is the problem"],"exampleFix":"// before: sticky enabled with dead backends\nsticky = true\n// after: ensure healthy instance exists, or fall back\nif !pickerHealthy(inst) { inst = cr.pickInstance() }","handlingStrategy":"retry","validationCode":"// health-check backends before relying on sticky routing\nfor _, s := range backends {\n    if !tcpReachable(s.addr) { log.Printf(\"backend down: %s\", s.addr) }\n}","typeGuard":null,"tryCatchPattern":"if resp.StatusCode == http.StatusServiceUnavailable {\n    return retryWithBackoff(func() error { return knock(relayURL, packet) })\n}","preventionTips":["Monitor backend NHP server health from the relay","Keep the relay peer table populated and reload on change","Alert on 503 rates from the relay endpoint"],"tags":["relay","load-balancing","availability"],"backgroundTag":"resource-not-found","analyzedSha":"6e04ca5ff03222a699c24205cd4bf8fee9af7ffe","analyzedAt":"2026-09-07T15:44:59.941Z","contentChangedAt":"2026-09-07T15:44:59.941Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}