huggingface/transformers · error · ValueError

No server currently running on {url}. To run a local server,

Error message

No server currently running on {url}. To run a local server, please run `transformers serve` in a separate shell. Find more information here: https://huggingface.co/docs/transformers/serving

What it means

check_health catches httpx.ConnectError while probing {service_root}/health. A ConnectError means nothing accepted the TCP connection at that address — no server is running there. The error advises starting `transformers serve` in another shell.

Source

Thrown at src/transformers/cli/chat.py:394

        # Check requirements
        if not is_rich_available():
            raise ImportError("You need to install rich to use the chat interface. (`pip install rich`)")

        # Run chat session
        asyncio.run(self._inner_run())

    @staticmethod
    def check_health(url):
        health_url = urljoin(get_service_root_url(url) + "/", "health")
        try:
            output = httpx.get(health_url)
            if output.status_code != 200:
                raise ValueError(
                    f"The server running on {url} returned status code {output.status_code} on health check (/health)."
                )
        except httpx.ConnectError:
            raise ValueError(
                f"No server currently running on {url}. To run a local server, please run `transformers serve` in a"
                f"separate shell. Find more information here: https://huggingface.co/docs/transformers/serving"
            )

        return True

    def handle_non_exit_user_commands(
        self,
        user_input: str,
        interface: RichInterface,
        examples: dict[str, dict[str, str]],
        config: GenerationConfig,
        chat: list[dict],
    ) -> tuple[list[dict], GenerationConfig]:
        """
        Handles all user commands except for `!exit`. May update the chat history (e.g. reset it) or the
        generation config (e.g. set a new flag).
        """

View on GitHub (pinned to a597f97485)

Solutions

Start the server: transformers serve --model_id <id> (default localhost:8000)
Pass the matching URL to chat: transformers chat --url http://localhost:<same-port>
If server and client are in different containers/hosts, bind the server to an reachable interface and use that address
Verify the port is listening: curl http://localhost:8000/health or ss -ltnp

Example fix

# before (server not started)
transformers chat  # ValueError: No server currently running

# after
# shell 1
transformers serve --model_id gpt2
# shell 2
transformers chat

Defensive patterns

Strategy: retry

Validate before calling

import socket

host, port = "localhost", 8000
with socket.create_connection((host, port), timeout=1):
    pass  # server reachable

Try / catch

import time, socket
for _ in range(60):
    try:
        with socket.create_connection(("localhost", 8000), timeout=1):
            break
    except OSError:
        time.sleep(1)
else:
    raise RuntimeError("transformers serve did not come up")

Prevention

Always start `transformers serve` before `transformers chat` (separate shells)
Pin the port explicitly on both sides: serve --port 8000, chat --url http://localhost:8000
In containers, bind serve to 0.0.0.0 and address it by service name/host IP, not localhost

When it happens

Trigger: Running `transformers chat` without having started `transformers serve`; wrong host/port; server bound to a different interface (localhost vs 0.0.0.0/LAN); firewall or Docker networking blocking the port; server already exited.

Common situations: Forgot to start the server; server started on a non-default port; inside a container trying to reach the host's server via localhost; typo in the URL; server crashed right after launch.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/4163a62f9ed801a2. Report an issue: GitHub.