unslothai/unsloth · error · RuntimeError

mlx-lm is too old for LRUPromptCache

Error message

mlx-lm is too old for LRUPromptCache

What it means

Raised by _MLXPromptCacheHistory.__init__ when _mlx_prompt_cache_api() returns None, i.e. the installed mlx-lm version does not expose the LRUPromptCache API surface (the prompt-cache constructor plus can_trim/trim helpers) that this backend's prefix-cache reuse depends on. The backend probes the installed mlx-lm for those symbols at construction; an older or refactored mlx-lm makes the probe fail, so the cache layer refuses to initialize rather than silently disabling prefix caching.

Source

Thrown at studio/backend/core/inference/mlx_inference.py:859

        if offset is None:
            return None
        if getattr(entry, "start_position", 0):
            return None
        window = getattr(entry, "max_size", None)
        if window is not None and offset > window:
            return None
        if covered is None:
            covered = offset
        elif covered != offset:
            return None
    return covered


class _MLXPromptCacheHistory:
    def __init__(self, max_entries, max_bytes):
        api = _mlx_prompt_cache_api()
        if api is None:
            raise RuntimeError("mlx-lm is too old for LRUPromptCache")
        lru_cls, make, can_trim, trim = api
        self._make_prompt_cache = make
        self._can_trim = can_trim
        self._trim = trim
        self._max_bytes = max_bytes
        self._lru = lru_cls(max_size = max_entries, max_bytes = max_bytes)

    def fetch(self, model, key, tokens):
        cache, rest = self._lru.fetch_nearest_cache(key, list(tokens))
        if cache is not None:
            if rest:
                return cache, list(rest)
            if self._can_trim(cache) and self._trim(cache, 1) == 1:
                return cache, list(tokens[-1:])
        if len(tokens) > 1:
            head = list(tokens[:-1])
            cache, rest = self._lru.fetch_nearest_cache(key, head)
            if cache is not None:

View on GitHub (pinned to 203007d190)

Solutions

1. Upgrade mlx-lm to the version required by this backend (e.g. pip install -U mlx-lm) and restart the inference process.
2. If you must stay on the old mlx-lm, disable the MLX prompt cache feature so _MLXPromptCacheHistory is never constructed.
3. Verify the API probe after upgrading: python -c "import mlx_lm, inspect; print(hasattr(mlx_lm, 'make_prompt_cache'))" or check for the LRU cache utilities.

Example fix

# before: old mlx-lm installed
cache = _MLXPromptCacheHistory(max_entries=8, max_bytes=1<<30)  # RuntimeError

# after
# pip install -U "mlx-lm>=0.2x"
cache = _MLXPromptCacheHistory(max_entries=8, max_bytes=1<<30)

if _mlx_prompt_cache_api() is None:
    logger.warning('mlx-lm too old for LRUPromptCache; prompt caching disabled')
    cache = None
else:
    cache = _MLXPromptCacheHistory(max_entries=8, max_bytes=1 << 30)

try:
    history = _MLXPromptCacheHistory(max_entries=n, max_bytes=b)
except RuntimeError as e:
    if 'too old' in str(e):
        history = None  # run without prefix caching
    else:
        raise

When it happens

Trigger: Constructing _MLXPromptCacheHistory (prompt caching enabled in the MLX backend) with an mlx-lm version older than the one that introduced LRUPromptCache / the make_prompt_cache + trim API, or a version where those internals were renamed.

Common situations: A stale virtualenv pinned to an old mlx-lm; upgrading unsloth/studio without upgrading mlx-lm; Apple Silicon boxes where mlx-lm was installed via an outdated brew/pip snapshot.

Related errors

• deadline reached before {method} {_redact_url(url)}
• NeMo Data Designer Hugging Face integration is not installed
• '{family_name}' needs diffusers ({pipeline_class}), but this
• Unsloth MLX: use_adapter must be None, True, False, or a str
• Unsloth MLX: cannot disable adapter layers without their bas

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/60df2df93b57bc37. Report an issue: GitHub.