NousResearch/hermes-agent · error · RuntimeError

Another iron-proxy start appears to be in progress (pidfile

Error message

Another iron-proxy start appears to be in progress (pidfile {pidfile} -> pid {existing_pid}).  Run `hermes egress stop` if that proxy is stuck.

What it means

_write_pidfile_safely opened with O_EXCL and got FileExistsError, and the existing pidfile resolves to a pid that _pid_alive() considers live. Since the top-of-start liveliness check should have caught this, reaching here means a genuine race with another concurrent start, or a recycled/foreign pid. The raise tells the operator to stop the existing instance.

Source

Thrown at agent/proxy_sources/iron_proxy.py:2029

    Side effect: also persists the in-process nonce to disk so
    cross-CLI-invocation ``_pid_alive`` checks (start in one process,
    stop in another) can still defeat PID recycling.
    """
    open_flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
    if hasattr(os, "O_NOFOLLOW"):
        open_flags |= os.O_NOFOLLOW
    try:
        fd = os.open(str(pidfile), open_flags, 0o600)
    except FileExistsError:
        # Pidfile already exists.  If it points at a live iron-proxy,
        # caller's _read_pid + _pid_alive at the top of start_proxy
        # should already have returned.  Reaching here means EITHER
        # the previous _pid_alive check raced (rare; another start in
        # flight), OR a stale pidfile survived a crash.  Discriminate
        # and retry once with O_TRUNC if stale.
        existing_pid = _read_pid()
        if existing_pid and _pid_alive(existing_pid):
            raise RuntimeError(
                f"Another iron-proxy start appears to be in progress "
                f"(pidfile {pidfile} -> pid {existing_pid}).  "
                f"Run `hermes egress stop` if that proxy is stuck."
            )
        # Stale — unlink and retry.
        try:
            pidfile.unlink()
        except FileNotFoundError:
            pass
        fd = os.open(str(pidfile), open_flags, 0o600)
    except OSError as exc:
        # ELOOP from a planted symlink at the pidfile path.
        raise RuntimeError(
            f"Refusing to write pidfile {pidfile}: {exc}.  "
            "Remove that path manually and retry."
        ) from exc

    try:

View on GitHub (pinned to c896c09c42)

Solutions

1. Run `hermes egress stop` to clear the live/stuck instance and its pidfile, then start again
2. If `stop` refuses because the pid is not really an iron-proxy (pid reuse), remove the pidfile manually and confirm no proxy is running (`ps -p <pid>`)
3. Serialize starts in automation: check `hermes egress status` before `start`, or take an external lock

Example fix

# before: concurrent starts
hermes egress start & hermes egress start &   # second raises 385
hermes egress stop && hermes egress start

# after: single clean start

# emulate the top-of-start check before invoking start_proxy:
from agent.proxy_sources import iron_proxy

def no_live_proxy(pidfile) -> bool:
    pid = iron_proxy._read_pid()
    return not (pid and iron_proxy._pid_alive(pid))

# if not no_live_proxy(...): run `hermes egress stop` instead of start

try:
    start_proxy(...)
except RuntimeError as e:
    if 'Another iron-proxy start' in str(e):
        stop_proxy()          # hermes egress stop
        start_proxy(...)      # one retry after clean stop

When it happens

Trigger: Two concurrent `hermes egress start` invocations (shell + cron, double Enter in a script); a start racing the top-of-function _read_pid/_pid_alive check; a stale pidfile whose number now belongs to any live unrelated process (pid reuse).

Common situations: A systemd unit and a manual start firing together; an automation script that starts without checking status; long uptime allowing pid-space wraparound so the stale pidfile's number matches an innocent process.

Related errors

• Refusing to write pidfile {pidfile}: {exc}. Remove that pat
• pidfile {pidfile} has unexpected owner uid={st.st_uid}
• An update is already in progress.
• iron-proxy log {log_path} has unexpected owner uid={st.st_ui
• failed to spawn iron-proxy: {exc}

AI-assisted analysis of NousResearch/hermes-agent@c896c09c42 (2026-08-14). Data as JSON: /api/errors/aa752b864e6c9b4b. Report an issue: GitHub.