What causes "failed to remove stale socket %s: %w" in joewalnes/websocketd?

Manually remove the stale socket as root or the owning user: `rm /run/app.sock`, then restart.

failed to remove stale socket %s: %w

Error message

failed to remove stale socket %s: %w

What it means

When the existing socket file does NOT respond to a dial (it's stale, left by an unclean shutdown), websocketd removes it before rebinding. If os.Remove fails — permission denied on the socket's directory, read-only filesystem, or the file vanished/reappeared — startup aborts with this wrapped error.

Source

Thrown at main.go:149

// serveUnixSocket removes a stale socket file left behind by an unclean
// shutdown (if any) and then serves on it. It only ever removes a path that
// is actually a socket, never an arbitrary file that happens to be there.
//
// A socket file at the path may equally belong to a server that is still
// running, so removing it unconditionally is not safe: unlinking a live
// server's socket and binding a new one in its place leaves that process
// running but permanently unreachable, with no error on either side. Probing
// first tells the two apart — a successful dial means someone is listening, a
// refused connection means the file is stale. Refusing to start on a live
// socket matches what a TCP listener already does when its port is taken.
func serveUnixSocket(path string, config *Config, log *libwebsocketd.LogScope) error {
	if info, err := os.Stat(path); err == nil && info.Mode()&os.ModeSocket != 0 {
		if conn, err := net.DialTimeout("unix", path, unixSocketProbeTimeout); err == nil {
			conn.Close()
			return fmt.Errorf("socket %s is already in use by a running server", path)
		}
		if err := os.Remove(path); err != nil {
			return fmt.Errorf("failed to remove stale socket %s: %w", path, err)
		}
	}
	return serve("unix", path, config, log)
}

// redirectAddress returns addr with its port replaced by redirPort. IPv6
// literals must be split with net.SplitHostPort (which understands brackets);
// splitting on the first colon lands inside "[::1]:port" and produced a
// malformed listener address that failed to bind — and, being a listener
// error, killed every other listener too.
func redirectAddress(addr string, redirPort int) (string, error) {
	host, _, err := net.SplitHostPort(addr)
	if err != nil {
		return "", fmt.Errorf("cannot derive redirect address from %q: %w", addr, err)
	}
	return net.JoinHostPort(host, strconv.Itoa(redirPort)), nil
}

View on GitHub (pinned to 7a8683dc7f)

Solutions

Manually remove the stale socket as root or the owning user: `rm /run/app.sock`, then restart.
Fix directory permissions: the process needs write+execute on the directory to unlink (chmod/chown the directory, not just the socket file).
Move the socket into a directory exclusively managed by the service user (e.g. /run/websocketd with RuntimeDirectory in systemd).
Check for immutable flags (lsattr/chattr -i) or read-only mounts if removal keeps failing.

Example fix

# before
websocketd --unixsocket=/var/run/app.sock ./handler  # EACCES removing stale socket
# after (systemd unit)
[Service]
RuntimeDirectory=websocketd
ExecStart=websocketd --unixsocket=/run/websocketd/app.sock ./handler

Defensive patterns

Strategy: retry

Validate before calling

const dir = filepath.Dir(socketPath)
if err := syscall.Access(dir, syscall.W_OK); err != nil {
    return fmt.Errorf("cannot remove stale socket: no write permission on %s: %w", dir, err)
}

Try / catch

if err := start(); err != nil && strings.Contains(err.Error(), "failed to remove stale socket") {
    os.Remove(sock) // best-effort manual cleanup, may need elevation
    err = start()
}

Prevention

Give the service exclusive ownership of the socket directory (systemd RuntimeDirectory).
Avoid read-only or shared mounts for socket paths.
Add a pre-start cleanup step running with sufficient privileges to unlink stale sockets.
Check for immutable attributes if manual rm also fails.

When it happens

Trigger: serveUnixSocket detects a stale socket (dial refused/timed out) then os.Remove(path) fails: process lacks write permission on the containing directory, the directory is read-only or a read-only mount, an immutable attribute is set, or a race replaces the file mid-remove.

Common situations: Old socket left in /var/run after a crash, now owned by a different uid; container filesystem remounted read-only; systemd-tmpfiles created the directory with restrictive modes; stale socket under a bind-mount owned by another container.

Related errors

AI-assisted analysis of joewalnes/websocketd@7a8683dc7f (2026-09-03). Data as JSON: /api/errors/c1c94454b5619b37. Report an issue: GitHub.