ArchiveBox/ArchiveBox · error · RuntimeError

Refusing to start a nested supervisord from inside a supervi

Error message

Refusing to start a nested supervisord from inside a supervisord-managed worker

What it means

Supervisord-managed workers already run under a supervisor, whose presence is signaled by the SUPERVISOR_SERVER_URL env var supervisord injects. start_new_supervisord_process refuses to spawn a nested supervisord in that case, since each worker would otherwise try to become its own daemon.

Source

Thrown at archivebox/workers/supervisord_util.py:920

                PID_FILE.unlink(missing_ok=True)
                SOCK_FILE.unlink(missing_ok=True)
        except (OSError, RuntimeError, ValueError) as err:
            _warn_background_cleanup("Could not clear owned supervisord pid/socket files", err)
        _supervisord_proc = None
    return True


def reap_foreground_supervisord_process() -> None:
    """Reap the supervisord child owned by this foreground parent if it exited."""

    global _supervisord_proc
    if _supervisord_proc and _supervisord_proc.poll() is not None:
        _supervisord_proc = None


def start_new_supervisord_process(daemonize=False):
    if os.environ.get("SUPERVISOR_SERVER_URL"):
        raise RuntimeError("Refusing to start a nested supervisord from inside a supervisord-managed worker")

    SOCK_FILE = get_sock_file()
    WORKERS_DIR = SOCK_FILE.parent / WORKERS_DIR_NAME
    LOG_FILE = CONSTANTS.LOGS_DIR / LOG_FILE_NAME
    CONFIG_FILE = SOCK_FILE.parent / CONFIG_FILE_NAME
    PID_FILE = SOCK_FILE.parent / PID_FILE_NAME
    stop_grace_seconds = configured_stopwaitsecs(tuple(_desired_supervisord_workers.values()))

    print(f"[🦸‍♂️] Supervisord starting{' in background' if daemonize else ''}...")
    pretty_log_path = pretty_path(LOG_FILE)
    print(f"    > Writing supervisord logs to: {pretty_log_path}")
    print(f"    > Writing task worker logs to: {pretty_log_path.replace('supervisord.log', 'worker_*.log')}")
    print(f"    > Using supervisord config file: {pretty_path(CONFIG_FILE)}")
    print(f"    > Using supervisord UNIX socket: {pretty_path(SOCK_FILE)}")
    print()

    # clear out existing stale state files
    shutil.rmtree(WORKERS_DIR, ignore_errors=True)

View on GitHub (pinned to 74564b2822)

Solutions

Use get_or_create_supervisord_process instead of start_new_supervisord_process so it connects to the inherited supervisor first
Ensure the inherited SUPERVISOR_SERVER_URL socket is reachable and exposes rpcinterface (supervisord.conf [unix_http_server]/[rpcinterface:supervisor]) — then the guard won't be hit via get_or_create path
Unset SUPERVISOR_SERVER_URL if this process should genuinely own a new supervisord and is not managed by one
Run the command outside of a supervisord-managed program (e.g. directly in a shell/systemd unit rather than as a supervisor program)

Example fix

// before: nested start inside a managed worker
env SUPERVISOR_SERVER_URL=... archivebox ... start_new_supervisord_process()
// after: connect-or-create via the safe entry point
supervisor = get_or_create_supervisord_process()

Defensive patterns

Strategy: validation

Validate before calling

import os
if os.environ.get('SUPERVISOR_SERVER_URL'):
    raise SystemExit('already inside a supervisord-managed worker; use get_or_create_supervisord_process (RPC) instead of starting a new daemon')

Prevention

Inside managed workers, always connect via supervisor RPC rather than spawning daemons
Do not define archivebox server/run as supervisord programs in external supervisor configs
Strip SUPERVISOR_SERVER_URL in wrappers/CI if the process must own its own daemon
Remember supervisord itself injects SUPERVISOR_SERVER_URL into children — any child must not re-spawn

When it happens

Trigger: Code invoked from within a supervisord-managed worker process (SUPERVISOR_SERVER_URL set) that calls start_new_supervisord_process directly — typically via get_or_create_supervisord_process — instead of connecting to the inherited supervisor over RPC.

Common situations: Running `archivebox server` or `archivebox run` inside a worker managed by an external/legacy supervisord; custom supervisor setups that set SUPERVISOR_SERVER_URL but whose socket ArchiveBox can't reach; CI wrappers that export supervisor variables.

Related errors

AI-assisted analysis of ArchiveBox/ArchiveBox@74564b2822 (2026-08-28). Data as JSON: /api/errors/94359e17e2081e2e. Report an issue: GitHub.