{"record":{"id":"e2d0766307725a87","repo":"NousResearch/hermes-agent","slug":"pidfile-pidfile-has-unexpected-owner-uid-st-st","errorCode":null,"errorMessage":"pidfile {pidfile} has unexpected owner uid={st.st_uid}","messagePattern":"pidfile (.+?) has unexpected owner uid=(.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"agent/proxy_sources/iron_proxy.py","lineNumber":2052,"sourceCode":"        # Stale — unlink and retry.\n        try:\n            pidfile.unlink()\n        except FileNotFoundError:\n            pass\n        fd = os.open(str(pidfile), open_flags, 0o600)\n    except OSError as exc:\n        # ELOOP from a planted symlink at the pidfile path.\n        raise RuntimeError(\n            f\"Refusing to write pidfile {pidfile}: {exc}.  \"\n            \"Remove that path manually and retry.\"\n        ) from exc\n\n    try:\n        # Ownership check — same st_uid pattern the log file uses.\n        try:\n            st = os.fstat(fd)\n            if hasattr(os, \"getuid\") and st.st_uid != os.getuid():\n                raise RuntimeError(\n                    f\"pidfile {pidfile} has unexpected owner uid={st.st_uid}\"\n                )\n        except AttributeError:\n            pass  # Windows\n        os.write(fd, str(pid).encode(\"utf-8\"))\n    finally:\n        os.close(fd)\n\n    # Persist the nonce next to the pidfile (sibling, 0o600).\n    # ``stop_proxy`` in a separate CLI invocation can read this and use\n    # it to confirm the pid still refers to our binary even though the\n    # module-global ``_proxy_nonce`` is fresh in the new process.\n    if _proxy_nonce:\n        noncefile = pidfile.with_suffix(\".nonce\")\n        nfd = -1\n        try:\n            nopen = os.O_WRONLY | os.O_CREAT | os.O_TRUNC\n            if hasattr(os, \"O_NOFOLLOW\"):","sourceCodeStart":2034,"sourceCodeEnd":2070,"githubUrl":"https://github.com/NousResearch/hermes-agent/blob/c896c09c42910c584c4c7d2325b58c14713ea42c/agent/proxy_sources/iron_proxy.py#L2034-L2070","documentation":"After successfully opening the pidfile fd, fstat shows st_uid differs from os.getuid() — the freshly created file is somehow owned by another user (setuid directory semantics, NFS root-squash id mapping, or the file pre-existed in a race window). Same ownership invariant the log file enforces; refuses to write the pid.","triggerScenarios":"_write_pidfile_safely on an NFS/fuse mount with different id mapping; a directory with setgid/sticky semantics that assigns unexpected ownership; a race where another user's process created the file between the EXCL open and the fstat. POSIX only.","commonSituations":"HERMES_HOME on NFS with root_squash/all_squash mismatching client uid; a shared/group-writable state dir where a group member's file lands first; containers with mismatching uid maps (userns) sharing a volume.","solutions":["Remove the foreign-owned pidfile: `rm <pidfile>` and retry the start","If on NFS/id-mapped mounts, move the proxy state dir to a native filesystem or fix the idmap domain so created files get your uid","Avoid sharing the HERMES_HOME/proxy state dir across users or containers with different uid maps"],"exampleFix":"# before: NFS squashed ownership\n# RuntimeError: pidfile ... unexpected owner uid=65534\nrm ~/.hermes/iron-proxy/proxy.pid\n# move state dir off NFS or fix idmap, then:\nhermes egress start\n\n# after: ownership check passes, pid written","handlingStrategy":"validation","validationCode":"import os\nfrom pathlib import Path\n\ndef state_dir_fs_safe(d: Path) -> bool:\n    # refuse NFS/fuse mounts whose idmap may squash ownership\n    fstype = os.statvfs(d).f_basetype if hasattr(os.statvfs, 'f_basetype') else ''\n    return 'nfs' not in fstype and d.stat().st_uid == os.getuid()\n\n# before start: require state_dir_fs_safe(proxy_state_dir)","typeGuard":null,"tryCatchPattern":"try:\n    start_proxy(...)\nexcept RuntimeError as e:\n    if 'unexpected owner uid' in str(e) and 'pidfile' in str(e):\n        pidfile.unlink(missing_ok=True)\n        # if it recurs, move the state dir off the id-mapped filesystem","preventionTips":["Keep the proxy state dir on a native filesystem (ext4/xfs/apfs), not NFS with squashing","Don't share HERMES_HOME across containers/users with different uid maps","After any restore, chown -R to the running uid before starting"],"tags":["filesystem","permissions","egress-proxy","pidfile","nfs"],"backgroundTag":null,"analyzedSha":"c896c09c42910c584c4c7d2325b58c14713ea42c","analyzedAt":"2026-08-14T17:18:01.089Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}