{"record":{"id":"ad13b9a95a8c8430","repo":"NousResearch/hermes-agent","slug":"refusing-to-write-pidfile-pidfile-exc-remov","errorCode":null,"errorMessage":"Refusing to write pidfile {pidfile}: {exc}.  Remove that path manually and retry.","messagePattern":"Refusing to write pidfile (.+?): (.+?)\\.  Remove that path manually and retry\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"agent/proxy_sources/iron_proxy.py","lineNumber":2042,"sourceCode":"        # the previous _pid_alive check raced (rare; another start in\n        # flight), OR a stale pidfile survived a crash.  Discriminate\n        # and retry once with O_TRUNC if stale.\n        existing_pid = _read_pid()\n        if existing_pid and _pid_alive(existing_pid):\n            raise RuntimeError(\n                f\"Another iron-proxy start appears to be in progress \"\n                f\"(pidfile {pidfile} -> pid {existing_pid}).  \"\n                f\"Run `hermes egress stop` if that proxy is stuck.\"\n            )\n        # Stale — unlink and retry.\n        try:\n            pidfile.unlink()\n        except FileNotFoundError:\n            pass\n        fd = os.open(str(pidfile), open_flags, 0o600)\n    except OSError as exc:\n        # ELOOP from a planted symlink at the pidfile path.\n        raise RuntimeError(\n            f\"Refusing to write pidfile {pidfile}: {exc}.  \"\n            \"Remove that path manually and retry.\"\n        ) from exc\n\n    try:\n        # Ownership check — same st_uid pattern the log file uses.\n        try:\n            st = os.fstat(fd)\n            if hasattr(os, \"getuid\") and st.st_uid != os.getuid():\n                raise RuntimeError(\n                    f\"pidfile {pidfile} has unexpected owner uid={st.st_uid}\"\n                )\n        except AttributeError:\n            pass  # Windows\n        os.write(fd, str(pid).encode(\"utf-8\"))\n    finally:\n        os.close(fd)\n","sourceCodeStart":2024,"sourceCodeEnd":2060,"githubUrl":"https://github.com/NousResearch/hermes-agent/blob/c896c09c42910c584c4c7d2325b58c14713ea42c/agent/proxy_sources/iron_proxy.py#L2024-L2060","documentation":"os.open of the pidfile path failed with an OSError other than FileExistsError — most notably ELOOP, which is what the kernel returns when O_NOFOLLOW hits a symlink at the pidfile path (the flag and message exist specifically to defeat a symlink plant). The raise refuses to write and tells the operator to remove the path manually.","triggerScenarios":"A symlink (or hard link to a sensitive file / path with too many symlink hops) exists where the pidfile should be created; also other open() failures like EACCES on a restrictive state dir.","commonSituations":"An attacker or misguided script symlinked the pidfile path at /etc/passwd-style targets; a leftover symlink from an experiment; HERMES_HOME/state dir permissions changed so open(O_CREAT) fails with EACCES.","solutions":["Inspect and remove the offending path: `ls -l <pidfile>` then `rm <pidfile>` (follow no symlinks: use `rm` on the link itself)","Fix state-dir permissions if the OSError text is 'Permission denied' (chown/chmod the parent dir)","Retry `hermes egress start`"],"exampleFix":"# before: pidfile path is a symlink\nls -l ~/.hermes/iron-proxy/proxy.pid  # -> /etc/some/target\nrm ~/.hermes/iron-proxy/proxy.pid\nhermes egress start\n\n# after: pidfile created as a regular 0600 file via O_EXCL|O_NOFOLLOW","handlingStrategy":"validation","validationCode":"from pathlib import Path\n\ndef pidfile_path_ok(p: Path) -> bool:\n    if p.is_symlink():\n        return False\n    if p.exists():\n        return p.owner() == __import__('getpass').getuser()  # or uid compare\n    return p.parent.is_dir()\n\n# before start: if not pidfile_path_ok(pidfile): inspect/remove it","typeGuard":"def is_regular_owned(p: Path) -> bool:\n    import os\n    try:\n        st = p.lstat()\n    except FileNotFoundError:\n        return True\n    import stat as S\n    return S.S_ISREG(st.st_mode) and st.st_uid == os.getuid()","tryCatchPattern":"try:\n    start_proxy(...)\nexcept RuntimeError as e:\n    if 'Refusing to write pidfile' in str(e):\n        pidfile.unlink(missing_ok=True)  # removes symlink itself, no follow\n        start_proxy(...)","preventionTips":["Treat any symlink at the pidfile path as hostile — remove the link, never write through it","Lock down the proxy state dir (0700, your uid) so others can't plant files","Alert on this error in monitoring: it can indicate a planted path, not just cruft"],"tags":["security","filesystem","egress-proxy","pidfile"],"backgroundTag":null,"analyzedSha":"c896c09c42910c584c4c7d2325b58c14713ea42c","analyzedAt":"2026-08-14T17:18:01.089Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}