NousResearch/hermes-agent · error · RuntimeError

pidfile {pidfile} has unexpected owner uid={st.st_uid}

Error message

pidfile {pidfile} has unexpected owner uid={st.st_uid}

What it means

After successfully opening the pidfile fd, fstat shows st_uid differs from os.getuid() — the freshly created file is somehow owned by another user (setuid directory semantics, NFS root-squash id mapping, or the file pre-existed in a race window). Same ownership invariant the log file enforces; refuses to write the pid.

Source

Thrown at agent/proxy_sources/iron_proxy.py:2052

        # Stale — unlink and retry.
        try:
            pidfile.unlink()
        except FileNotFoundError:
            pass
        fd = os.open(str(pidfile), open_flags, 0o600)
    except OSError as exc:
        # ELOOP from a planted symlink at the pidfile path.
        raise RuntimeError(
            f"Refusing to write pidfile {pidfile}: {exc}.  "
            "Remove that path manually and retry."
        ) from exc

    try:
        # Ownership check — same st_uid pattern the log file uses.
        try:
            st = os.fstat(fd)
            if hasattr(os, "getuid") and st.st_uid != os.getuid():
                raise RuntimeError(
                    f"pidfile {pidfile} has unexpected owner uid={st.st_uid}"
                )
        except AttributeError:
            pass  # Windows
        os.write(fd, str(pid).encode("utf-8"))
    finally:
        os.close(fd)

    # Persist the nonce next to the pidfile (sibling, 0o600).
    # ``stop_proxy`` in a separate CLI invocation can read this and use
    # it to confirm the pid still refers to our binary even though the
    # module-global ``_proxy_nonce`` is fresh in the new process.
    if _proxy_nonce:
        noncefile = pidfile.with_suffix(".nonce")
        nfd = -1
        try:
            nopen = os.O_WRONLY | os.O_CREAT | os.O_TRUNC
            if hasattr(os, "O_NOFOLLOW"):

View on GitHub (pinned to c896c09c42)

Solutions

1. Remove the foreign-owned pidfile: `rm <pidfile>` and retry the start
2. If on NFS/id-mapped mounts, move the proxy state dir to a native filesystem or fix the idmap domain so created files get your uid
3. Avoid sharing the HERMES_HOME/proxy state dir across users or containers with different uid maps

Example fix

# before: NFS squashed ownership
# RuntimeError: pidfile ... unexpected owner uid=65534
rm ~/.hermes/iron-proxy/proxy.pid
# move state dir off NFS or fix idmap, then:
hermes egress start

# after: ownership check passes, pid written

import os
from pathlib import Path

def state_dir_fs_safe(d: Path) -> bool:
    # refuse NFS/fuse mounts whose idmap may squash ownership
    fstype = os.statvfs(d).f_basetype if hasattr(os.statvfs, 'f_basetype') else ''
    return 'nfs' not in fstype and d.stat().st_uid == os.getuid()

# before start: require state_dir_fs_safe(proxy_state_dir)

try:
    start_proxy(...)
except RuntimeError as e:
    if 'unexpected owner uid' in str(e) and 'pidfile' in str(e):
        pidfile.unlink(missing_ok=True)
        # if it recurs, move the state dir off the id-mapped filesystem

When it happens

Trigger: _write_pidfile_safely on an NFS/fuse mount with different id mapping; a directory with setgid/sticky semantics that assigns unexpected ownership; a race where another user's process created the file between the EXCL open and the fstat. POSIX only.

Common situations: HERMES_HOME on NFS with root_squash/all_squash mismatching client uid; a shared/group-writable state dir where a group member's file lands first; containers with mismatching uid maps (userns) sharing a volume.

Related errors

• iron-proxy log {log_path} has unexpected owner uid={st.st_ui
• Refusing to write pidfile {pidfile}: {exc}. Remove that pat
• Hermes backend did not become ready: ${detail}
• Could not create directory: ${error.message}
• Unsafe SSH control dir: ${controlDir} is owned by uid ${st.u

AI-assisted analysis of NousResearch/hermes-agent@c896c09c42 (2026-08-14). Data as JSON: /api/errors/e2d0766307725a87. Report an issue: GitHub.