juanfont/headscale · warning

health check timed out

Error message

health check timed out

What it means

Returned as HTTP 403 when a request to a headscale debug endpoint arrives from a client that is neither a tailnet node (verified via the ts2021 noise-authenticated 'headscale' node) nor a private/LAN IP. The debug router deliberately gates diagnostics (ping, map dumps, etc.) so they are not exposed to the public internet. The check parses r.RemoteAddr and allows the request only when the source IP is RFC1918-private or the caller is inside the tailnet.

Source

Thrown at cmd/dev/main.go:27

	"flag"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/exec"
	"os/signal"
	"path/filepath"
	"strconv"
	"syscall"
	"time"
)

var (
	port = flag.Int("port", 8080, "headscale listen port")
	keep = flag.Bool("keep", false, "keep state directory on exit")
)

var errHealthTimeout = errors.New("health check timed out")

var errEmptyAuthKey = errors.New("empty auth key in response")

// maxDevPort is the highest --port value that keeps the derived metrics
// port (port+1010) inside the valid 1..65535 TCP range.
const maxDevPort = 64525

const devConfig = `---
server_url: http://127.0.0.1:%d
listen_addr: 127.0.0.1:%d
metrics_listen_addr: 127.0.0.1:%d

noise:
  private_key_path: %s/noise_private.key

prefixes:
  v4: 100.64.0.0/10
  v6: fd7a:115c:a1e0::/48

View on GitHub (pinned to 565fd254d0)

Solutions

1. Access the debug endpoint from a machine inside the tailnet (the authenticated 'headscale' node path), e.g. curl from a tailscale-up node or via the headscale host's tailscale IP.
2. If behind a reverse proxy, configure it to pass the real client IP (X-Forwarded-For handling / proxy_protocol) so RemoteAddr resolves to your private LAN address, or reach headscale directly on the LAN.
3. SSH-tunnel or wireguard into the private network first, then curl the endpoint from there (source IP becomes private and passes ip.IsPrivate()).
4. If you truly need remote debug access, expose headscale only over a VPN/tailscale and never widen this check to public IPs.

Example fix

# before (from a public client IP -> 403 debug access denied)
curl http://headscale.example.com/debug

# after (from inside the tailnet / private network)
tailscale up
curl http://<headscale-lan-or-tailscale-ip>/debug

// Before hitting /debug/*, confirm the source IP is private or you are on tailnet.
conn, err := net.Dial("udp", "8.8.8.8:80") // learn egress iface without sending traffic
if err == nil {
    defer conn.Close()
    localIP := conn.LocalAddr().(*net.UDPAddr).IP
    if !localIP.IsPrivate() {
        log.Printf("source %s is not private; debug endpoints will 403 — connect via tailnet first", localIP)
    }
}

When it happens

Trigger: Curling /debug/* endpoints from a machine whose source IP is public (e.g. operator's home IP over the internet, or behind a reverse proxy that forwards the proxy's public IP instead of the real client IP), or from a non-private address that is not part of the tailnet. Also triggered when a reverse proxy rewrites RemoteAddr to a non-private address.

Common situations: Running headscale behind a reverse proxy (nginx/caddy/traefik) on a public VPS and trying to open /debug from a browser at home; running the proxy in a Docker network where the forwarded address is the container bridge IP but the operator tests through a public hostname; forgetting that the allowlist is private-IP-or-tailnet only.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• empty auth key in response
• MOCKOIDC_CLIENT_SECRET not defined
• test pattern is required as first argument or use --test fla
• directory is required
• creating node: %w

AI-assisted analysis of juanfont/headscale@565fd254d0 (2026-08-15). Data as JSON: /api/errors/3ad3d4e49b138c42. Report an issue: GitHub.