ArchiveBox/ArchiveBox · error · SystemExit

[X] Your system is running python3 scripts with a bad locale

Error message

[X] Your system is running python3 scripts with a bad locale setting: {PYTHON_ENCODING} (it should be UTF-8).

What it means

check_io_encoding verifies the active stdout encoding is UTF-8, since ArchiveBox archives pages with arbitrary Unicode. If the resolved encoding (from sys.__stdout__/stdout/__stderr__/stderr) is anything other than UTF-8, it prints the offending encoding and fix instructions, then raises SystemExit(2).

Source

Thrown at archivebox/misc/checks.py:182

        if blocking:
            raise SystemExit(3)
    return pending


def check_io_encoding():
    PYTHON_ENCODING = (sys.__stdout__ or sys.stdout or sys.__stderr__ or sys.stderr).encoding.upper().replace("UTF8", "UTF-8")

    if PYTHON_ENCODING != "UTF-8":
        print(
            f"[red][X] Your system is running python3 scripts with a bad locale setting: {PYTHON_ENCODING} (it should be UTF-8).[/red]",
            file=sys.stderr,
        )
        print('    To fix it, add the line "export PYTHONIOENCODING=UTF-8" to your ~/.bashrc file (without quotes)', file=sys.stderr)
        print('    Or if you\'re using ubuntu/debian, run "dpkg-reconfigure locales"', file=sys.stderr)
        print("")
        print("    Confirm that it's fixed by opening a new shell and running:", file=sys.stderr)
        print('        python3 -c "import sys; print(sys.stdout.encoding)"   # should output UTF-8', file=sys.stderr)
        raise SystemExit(2)


def check_not_root():
    is_getting_help = "-h" in sys.argv or "--help" in sys.argv or "help" in sys.argv
    is_getting_version = "--version" in sys.argv or "version" in sys.argv

    if os.geteuid() == 0 and not (is_getting_help or is_getting_version):
        print("[yellow][!] Running ArchiveBox as root is not recommended.[/yellow]", file=sys.stderr)
        print("    Root-owned DATA_DIR files may be inaccessible to non-root users later.", file=sys.stderr)
        print("        https://github.com/ArchiveBox/ArchiveBox/wiki/Security-Overview#do-not-run-as-root", file=sys.stderr)


def is_archivebox_source_root(path: Path | str | None = None) -> bool:
    """Return whether path is the root of an ArchiveBox source checkout."""
    path = Path(path or os.getcwd()).resolve()
    try:
        return (path / ".git").exists() and (path / "archivebox" / "__init__.py").is_file() and (path / "pyproject.toml").is_file()
    except OSError:

View on GitHub (pinned to 74564b2822)

Solutions

  1. Add `export PYTHONIOENCODING=UTF-8` to your shell profile / cron / systemd unit environment
  2. Set proper locale: `export LANG=C.UTF-8` (or `dpkg-reconfigure locales` on Debian/Ubuntu and choose a UTF-8 locale)
  3. Fix it in Docker by setting ENV LANG=C.UTF-8 in the image or `environment:` in compose
  4. Verify with `python3 -c "import sys; print(sys.stdout.encoding)"` in a new shell

Example fix

# before (crontab)
0 * * * * archivebox update
# after
0 * * * * PYTHONIOENCODING=UTF-8 LANG=C.UTF-8 archivebox update
Defensive patterns

Strategy: validation

Validate before calling

import sys, os
enc = (sys.__stdout__ or sys.stdout).encoding or ""
if enc.upper().replace("UTF8", "UTF-8") != "UTF-8":
    os.environ["PYTHONIOENCODING"] = "UTF-8"
    os.execv(sys.executable, [sys.executable] + sys.argv)  # relaunch with UTF-8

Type guard

def has_utf8_stdio() -> bool:
    import sys
    enc = (sys.__stdout__ or sys.stdout).encoding
    return bool(enc) and enc.upper().replace("UTF8", "UTF-8") == "UTF-8"

Try / catch

try:
    run_archivebox_cmd(cmd)
except SystemExit as e:
    if e.code == 2 and "bad locale setting" in last_stderr():
        env = {**os.environ, "PYTHONIOENCODING": "UTF-8", "LANG": "C.UTF-8"}
        subprocess.run(cmd, env=env, check=True)
    else:
        raise

Prevention

When it happens

Trigger: Running archivebox in an environment where Python's stdio encoding is non-UTF-8: POSIX/C locale over SSH, cron/systemd with minimal LANG, containers without locale set, or piping output in a shell whose locale is ASCII.

Common situations: SSH into a server with no LANG/LC_ALL set; cron jobs inheriting empty environment; Docker images lacking locales package; CI runners with C locale; older Debian/Ubuntu defaulting to ASCII.

Related errors

AI-assisted analysis of ArchiveBox/ArchiveBox@74564b2822 (2026-08-28). Data as JSON: /api/errors/40230a07a63bf6e0. Report an issue: GitHub.