NousResearch/hermes-agent · error · RuntimeError

failed to spawn iron-proxy: {exc}

Error message

failed to spawn iron-proxy: {exc}

What it means

subprocess.Popen raised OSError while launching the iron-proxy binary with ['-config', cfg]. This means the OS refused to exec: the binary does not exist at bin_path, is not executable, has a bad ELF interpreter, or resource limits (EMFILE/ENOMEM) were hit. The log fd is closed and the error is wrapped with the underlying OSError text.

Source

Thrown at agent/proxy_sources/iron_proxy.py:1883

        # into the child so we can close ours unconditionally below.
        # NOTE: on Windows ``start_new_session`` is invalid; we don't
        # support Windows for the proxy (the binary itself doesn't ship)
        # but the kwarg is POSIX-only and silently ignored on Win.
        popen_kwargs: Dict = dict(
            env=env,
            stdin=subprocess.DEVNULL,
            stdout=log_fd,
            stderr=subprocess.STDOUT,
        )
        if platform.system() != "Windows":
            popen_kwargs["start_new_session"] = True
        proc = subprocess.Popen(  # noqa: S603 — binary path is trusted
            [str(bin_path), "-config", str(cfg)],
            **popen_kwargs,
        )
    except OSError as exc:
        os.close(log_fd)
        raise RuntimeError(f"failed to spawn iron-proxy: {exc}") from exc
    finally:
        # Close our copy of the fd whether Popen raised or succeeded.
        # The child has its own dup via Popen, so it's still writing.
        try:
            os.close(log_fd)
        except OSError:
            pass

    # Write the pidfile IMMEDIATELY after Popen, BEFORE the listening
    # verification.  If the parent dies during the poll loop (SIGINT,
    # OOM, kernel pause), the pidfile is still on disk so the next
    # ``hermes egress stop`` can clean up the orphan.  Failure paths
    # below unlink the pidfile when they kill the child.
    pidfile = _pidfile()
    try:
        _write_pidfile_safely(pidfile, proc.pid)
    except RuntimeError:
        # Kill the orphan so we don't leave a daemon nobody can stop.

View on GitHub (pinned to c896c09c42)

Solutions

1. Check the OSError detail in the message; if 'No such file or directory', reinstall the proxy binary (re-run `hermes egress setup` or the auto-install path) so bin_path exists
2. If 'Permission denied', chmod +x the binary or fix mount options (noexec)
3. If 'Exec format error', the binary is for the wrong arch/OS — reinstall the correct build
4. If EMFILE/ENOMEM, reduce load / raise ulimits and retry

Example fix

# before: bin missing after venv recreate
ls -l ~/.hermes/iron-proxy/iron-proxy  # missing
hermes egress setup   # re-installs the binary
hermes egress start

# after: Popen succeeds, startup poll begins

import os
from pathlib import Path

def binary_spawnable(bin_path: Path) -> bool:
    return bin_path.is_file() and os.access(bin_path, os.X_OK)

# guard before start_proxy():
# if not binary_spawnable(bin_path): reinstall / re-run hermes egress setup

try:
    start_proxy(...)
except RuntimeError as e:
    if 'failed to spawn' in str(e):
        reinstall_proxy_binary()  # hermes egress setup / auto-install
        start_proxy(...)  # single retry after remediation

When it happens

Trigger: start_proxy() with a missing or non-executable iron-proxy binary at the resolved bin_path; binary built for a different architecture/libc; exec format error on a corrupted download; file-descriptor or memory exhaustion at spawn time.

Common situations: A partial/failed proxy auto-install left a truncated binary; the binary was deleted by a cleanup tool or antivirus; running on an architecture the binary was not built for; bin_path pointing into a venv that was recreated; disk-full corrupted the binary.

Related errors

• LSP server binary not found: {cmd[0]} ({e})
• cannot send {method!r}: stdin closed
• send failed for {method!r}: {e}
• unexpected EOF while reading LSP headers (partial={e.partial
• truncated LSP body: expected {n} bytes, got {len(e.partial)}

AI-assisted analysis of NousResearch/hermes-agent@c896c09c42 (2026-08-14). Data as JSON: /api/errors/54ea4d1110c4d100. Report an issue: GitHub.