headroomlabs-ai/headroom · error · TiktokenLoadError

tiktoken encoding {encoding_name!r} previously failed to loa

Error message

tiktoken encoding {encoding_name!r} previously failed to load

What it means

TiktokenLoadError raised in fail-fast form: the encoding name is in the module-level _load_failed set, meaning a previous _get_encoding() call for it timed out (stalled vocab download, GH #956) and was blacklisted so every later request fails immediately instead of re-blocking the worker thread. It persists for the life of the process.

Source

Thrown at headroom/tokenizers/tiktoken_counter.py:120

# Default encoding for unknown models
DEFAULT_ENCODING = "cl100k_base"


@lru_cache(maxsize=8)
def _get_encoding(encoding_name: str):
    """Get a tiktoken encoding, cached for performance.

    Bounded by ``HEADROOM_TIKTOKEN_LOAD_TIMEOUT_SECONDS`` (default 10s): tiktoken's
    vocab download has no network timeout, so we run the load on a worker thread
    and raise :class:`TiktokenLoadError` if it doesn't finish in time, letting
    callers fall back to estimation rather than hang the request (GH #956). The
    first timed-out encoding is remembered so later calls fail fast instead of
    re-blocking on every request.
    """
    import tiktoken

    if encoding_name in _load_failed:
        raise TiktokenLoadError(f"tiktoken encoding {encoding_name!r} previously failed to load")

    box: dict[str, Any] = {}

    def _load() -> None:
        try:
            box["enc"] = tiktoken.get_encoding(encoding_name)
        except BaseException as exc:  # noqa: BLE001 - re-raised in the calling thread
            box["err"] = exc

    worker = threading.Thread(target=_load, name=f"tiktoken-load-{encoding_name}", daemon=True)
    worker.start()
    worker.join(_load_timeout_seconds())

    if worker.is_alive():
        _load_failed.add(encoding_name)
        logger.warning(
            "tiktoken encoding %r did not load within %.1fs (likely a stalled vocab "
            "download); falling back to token estimation. Pre-populate TIKTOKEN_CACHE_DIR "

View on GitHub (pinned to 322425c43b)

Solutions

1. Fix the underlying load condition, then restart the process — the blacklist is memory-only and resets.
2. Pre-populate TIKTOKEN_CACHE_DIR (download the vocab in a build step or bake it into the image) so the load is instant and never times out.
3. Raise HEADROOM_TIKTOKEN_LOAD_TIMEOUT_SECONDS if your network is just slow, not blocked.
4. Catch TiktokenLoadError and use the estimating fallback while scheduling a process restart.

Example fix

# before
# first request timed out downloading cl100k_base; later requests:
enc = _get_encoding("cl100k_base")  # TiktokenLoadError: previously failed

# after (deploy step): export TIKTOKEN_CACHE_DIR=/opt/tiktoken_cache && python -c "import tiktoken; tiktoken.get_encoding('cl100k_base')"
# then restart the app process; loads hit the warm cache

from headroom.tokenizers.tiktoken_counter import _load_failed, TiktokenLoadError
if name in _load_failed:
    use_estimator_or_restart()  # fail-fast is guaranteed for this process

def tiktoken_usable(name: str) -> bool:
    from headroom.tokenizers.tiktoken_counter import _load_failed
    return name not in _load_failed

from headroom.tokenizers.tiktoken_counter import TiktokenLoadError
try:
    enc = load_encoding(name)
except TiktokenLoadError:
    enc = None  # use EstimatingTokenCounter; schedule process restart after cache fix

When it happens

Trigger: First load of e.g. 'cl100k_base' exceeded HEADROOM_TIKTOKEN_LOAD_TIMEOUT_SECONDS (default 10s) → recorded in _load_failed; any subsequent get_encoding/counter call for that name in the same process raises immediately.

Common situations: A request early in the process hit a stalled download (proxy/firewall); after fixing the environment the process still refuses to load because of the in-memory blacklist; long-lived servers that never restart after one transient network failure.

Related errors

• tiktoken encoding {encoding_name!r} load timed out
• upstream request failed: {0}
• `{' '.join(cmd)}` produced no output for {idle_cap}s. Check
• headroom proxy was not ready at {url}: {last_error}
• Encoding not available for {self.model} - tokenizer {self.to

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/2b964ed7f433bd7f. Report an issue: GitHub.