What causes "codex app-server stdin closed unexpectedly: {exc}" in NousResearch/hermes-agent?

Capture and log the subprocess's stderr — it usually contains the reason codex exited.

NousResearch/hermes-agent · error · RuntimeError

codex app-server stdin closed unexpectedly: {exc}

Error message

codex app-server stdin closed unexpectedly: {exc}

What it means

RuntimeError from _send() (agent/transports/codex_app_server.py:310) wrapping BrokenPipeError/ValueError raised while writing a JSON-RPC frame to the app-server's stdin. It means the write side is gone in practice: the codex subprocess has exited (or closed stdin), so the pipe is broken. ValueError is included because writing to a closed file object raises it.

Source

Thrown at agent/transports/codex_app_server.py:310

    def _take_id(self) -> int:
        # JSON-RPC ids only need to be unique per-connection. A simple
        # monotonically increasing int is the common choice and matches what
        # codex's own clients use.
        rid = self._next_id
        self._next_id += 1
        return rid

    def _send(self, obj: dict) -> None:
        if self._closed:
            raise RuntimeError("codex app-server client is closed")
        if self._proc.stdin is None:
            raise RuntimeError("codex app-server stdin not available")
        try:
            self._proc.stdin.write((json.dumps(obj) + "\n").encode("utf-8"))
            self._proc.stdin.flush()
        except (BrokenPipeError, ValueError) as exc:
            raise RuntimeError(
                f"codex app-server stdin closed unexpectedly: {exc}"
            ) from exc

    def _read_stdout(self) -> None:
        if self._proc.stdout is None:
            return
        try:
            for line in iter(self._proc.stdout.readline, b""):
                if not line:
                    break
                line = line.strip()
                if not line:
                    continue
                try:
                    msg = json.loads(line)
                except json.JSONDecodeError:
                    # Non-JSON output is unexpected on stdout; tracing belongs
                    # on stderr. Surface it via stderr buffer for diagnostics.

View on GitHub (pinned to c896c09c42)

Solutions

Capture and log the subprocess's stderr — it usually contains the reason codex exited.
Check _proc.poll() after this error; a non-None exit code confirms the child died and the client must be recreated.
Validate the codex command/version before spawning (bad flags cause immediate exit).
Treat this error as terminal for the connection: tear down and respawn rather than retrying sends.

Example fix

# before
try:
    client.request("thread/start", params)
except RuntimeError:
    retry()  # pointless: the child is dead

# after
try:
    client.request("thread/start", params)
except RuntimeError as exc:
    if client._proc.poll() is not None:
        raise RuntimeError(
            f"codex exited rc={client._proc.returncode}: "
            f"{client._proc.stderr.read().decode(errors='replace')[:500]}"
        ) from exc
    raise

Defensive patterns

Strategy: try-catch

Validate before calling

def child_healthy(proc) -> bool:
    return proc.poll() is None and proc.stdin is not None and not proc.stdin.closed

Try / catch

try:
    client.request(method, params)
except RuntimeError as exc:
    if "stdin closed unexpectedly" in str(exc):
        stderr = proc.stderr.read().decode(errors="replace") if proc.stderr else ""
        raise RuntimeError(f"codex exited (rc={proc.poll()}): {stderr[:500]}") from exc
    raise

Prevention

Capture the child's stderr into a log from the start — it explains most exits.
Check proc.poll() after any transport error; a dead child means recreate, not retry.
Validate the codex command/version before spawn to avoid instant exits.

When it happens

Trigger: The codex app-server process crashed or exited (bad args, panic, OOM-kill, explicit quit) while the client still had queued requests; sends from a background thread after the reader observed EOF; flushing after the child died mid-request.

Common situations: Codex binary mismatch making the process exit at startup; the child killed by the OS; a close() race where another writer already flushed; diagnosing this usually reveals an earlier stderr message from codex that was swallowed.

Related errors

AI-assisted analysis of NousResearch/hermes-agent@c896c09c42 (2026-08-14). Data as JSON: /api/errors/14c9bd01dccdb843. Report an issue: GitHub.