cloudflare/cloudflared · critical

stack trace: %s

Error message

stack trace: %s

What it means

ServeTunnel wraps its work in a recover() handler so panics inside the tunnel serving loop do not crash the daemon. When a panic occurs, the recovered value is converted to an error and wrapped with the goroutine stack trace under the message 'stack trace: %s'; recoverable is set so the connection is re-established.

Source

Thrown at supervisor/tunnel.go:386

// on error returns a flag indicating if error can be retried
func (e *EdgeTunnelServer) serveTunnel(
	ctx context.Context,
	connLog *ConnAwareLogger,
	addr *allregions.EdgeAddr,
	connIndex uint8,
	fuse *booleanFuse,
	backoff *protocolFallback,
	protocol connection.Protocol,
) (err error, recoverable bool) {
	// Treat panics as recoverable errors
	defer func() {
		if r := recover(); r != nil {
			var ok bool
			err, ok = r.(error)
			if !ok {
				err = fmt.Errorf("ServeTunnel: %v", r)
			}
			err = errors.Wrapf(err, "stack trace: %s", string(debug.Stack()))
			recoverable = true
		}
	}()

	defer e.config.Observer.SendDisconnect(connIndex)
	err, recoverable = e.serveConnection(
		ctx,
		connLog,
		addr,
		connIndex,
		fuse,
		backoff,
		protocol,
	)

	if err != nil {
		switch err := err.(type) {
		case connection.DupConnRegisterTunnelError:

View on GitHub (pinned to 2253eeeb25)

Solutions

1. Read the wrapped stack trace in the error to locate the panicking frame
2. Update cloudflared — many panic bugs are fixed in newer releases
3. Look for concurrent mutation of shared config/state and add synchronization (mutex or Clone before mutate)
4. Report the stack trace to cloudflared issues if it reproduces

Example fix

// before
cfg.CurvePreferences = curves // mutating shared *tls.Config -> data race/panic
// after
cfg = cfg.Clone()
cfg.CurvePreferences = curves

defer func() {
    if r := recover(); r != nil {
        err = fmt.Errorf("ServeTunnel: %v", r)
        err = errors.Wrapf(err, "stack trace: %s", string(debug.Stack()))
        recoverable = true
    }
}()

When it happens

Trigger: Any panic inside serveConnection/handleTunnel: nil pointer dereference, index out of range, etc., while serving a tunnel connection on a specific connIndex.

Common situations: Race conditions corrupting shared state (e.g. unsynchronized *tls.Config mutation); unexpected nil fields from edge; bugs triggered by malformed edge messages.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• ServeTunnel: %v
• ErrTunnelNameConflict
• tunnel name required
• you cannot use UUIDs as tunnel names
• ErrNoTunnelID

AI-assisted analysis of cloudflare/cloudflared@2253eeeb25 (2026-09-06). Data as JSON: /api/errors/72876f10d08f8146. Report an issue: GitHub.