{"record":{"id":"60df2df93b57bc37","repo":"unslothai/unsloth","slug":"mlx-lm-is-too-old-for-lrupromptcache","errorCode":null,"errorMessage":"mlx-lm is too old for LRUPromptCache","messagePattern":"mlx-lm is too old for LRUPromptCache","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/mlx_inference.py","lineNumber":859,"sourceCode":"        if offset is None:\n            return None\n        if getattr(entry, \"start_position\", 0):\n            return None\n        window = getattr(entry, \"max_size\", None)\n        if window is not None and offset > window:\n            return None\n        if covered is None:\n            covered = offset\n        elif covered != offset:\n            return None\n    return covered\n\n\nclass _MLXPromptCacheHistory:\n    def __init__(self, max_entries, max_bytes):\n        api = _mlx_prompt_cache_api()\n        if api is None:\n            raise RuntimeError(\"mlx-lm is too old for LRUPromptCache\")\n        lru_cls, make, can_trim, trim = api\n        self._make_prompt_cache = make\n        self._can_trim = can_trim\n        self._trim = trim\n        self._max_bytes = max_bytes\n        self._lru = lru_cls(max_size = max_entries, max_bytes = max_bytes)\n\n    def fetch(self, model, key, tokens):\n        cache, rest = self._lru.fetch_nearest_cache(key, list(tokens))\n        if cache is not None:\n            if rest:\n                return cache, list(rest)\n            if self._can_trim(cache) and self._trim(cache, 1) == 1:\n                return cache, list(tokens[-1:])\n        if len(tokens) > 1:\n            head = list(tokens[:-1])\n            cache, rest = self._lru.fetch_nearest_cache(key, head)\n            if cache is not None:","sourceCodeStart":841,"sourceCodeEnd":877,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/mlx_inference.py#L841-L877","documentation":"Raised by _MLXPromptCacheHistory.__init__ when _mlx_prompt_cache_api() returns None, i.e. the installed mlx-lm version does not expose the LRUPromptCache API surface (the prompt-cache constructor plus can_trim/trim helpers) that this backend's prefix-cache reuse depends on. The backend probes the installed mlx-lm for those symbols at construction; an older or refactored mlx-lm makes the probe fail, so the cache layer refuses to initialize rather than silently disabling prefix caching.","triggerScenarios":"Constructing _MLXPromptCacheHistory (prompt caching enabled in the MLX backend) with an mlx-lm version older than the one that introduced LRUPromptCache / the make_prompt_cache + trim API, or a version where those internals were renamed.","commonSituations":"A stale virtualenv pinned to an old mlx-lm; upgrading unsloth/studio without upgrading mlx-lm; Apple Silicon boxes where mlx-lm was installed via an outdated brew/pip snapshot.","solutions":["Upgrade mlx-lm to the version required by this backend (e.g. pip install -U mlx-lm) and restart the inference process.","If you must stay on the old mlx-lm, disable the MLX prompt cache feature so _MLXPromptCacheHistory is never constructed.","Verify the API probe after upgrading: python -c \"import mlx_lm, inspect; print(hasattr(mlx_lm, 'make_prompt_cache'))\" or check for the LRU cache utilities."],"exampleFix":"# before: old mlx-lm installed\ncache = _MLXPromptCacheHistory(max_entries=8, max_bytes=1<<30)  # RuntimeError\n\n# after\n# pip install -U \"mlx-lm>=0.2x\"\ncache = _MLXPromptCacheHistory(max_entries=8, max_bytes=1<<30)","handlingStrategy":"validation","validationCode":"if _mlx_prompt_cache_api() is None:\n    logger.warning('mlx-lm too old for LRUPromptCache; prompt caching disabled')\n    cache = None\nelse:\n    cache = _MLXPromptCacheHistory(max_entries=8, max_bytes=1 << 30)","typeGuard":null,"tryCatchPattern":"try:\n    history = _MLXPromptCacheHistory(max_entries=n, max_bytes=b)\nexcept RuntimeError as e:\n    if 'too old' in str(e):\n        history = None  # run without prefix caching\n    else:\n        raise","preventionTips":["Pin mlx-lm to the minimum version stated by the backend in your lockfile.","Run a startup feature-probe (same as _mlx_prompt_cache_api) and degrade gracefully to no-cache instead of crashing.","Include mlx-lm version in startup logs to diagnose environment drift quickly."],"tags":["mlx","version-mismatch","prompt-cache","dependencies"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}