Error reading persistent compilation cache entry for '{cache

Error message

Error reading persistent compilation cache entry for '{cache_key}': {type(ex).__name__}: {ex}

What it means

JAX persisted a compiled executable in a persistent compilation cache (jax_compilation_cache). While checking whether an executable exists for a cache key, the cache read failed (e.g. corrupt entry, unreadable GCS/S3 bucket, deserialization mismatch). JAX downgrades this to a warning and recompiles from scratch.

Source

Thrown at jax/_src/compiler.py:798


def _should_raise_persistent_cache_error(ex: Exception) -> bool:
  """Returns True if the exception should be raised, False if it should be warned."""
  return (
      config.raise_persistent_cache_errors.value or
      isinstance(ex, compilation_cache.CacheVerificationError)
  )


def _is_executable_in_cache(backend, cache_key) -> bool:
  """Checks if executable is presented in cache on a given key
  """
  try:
    return compilation_cache.is_executable_in_cache(backend, cache_key)
  except Exception as ex:
    if _should_raise_persistent_cache_error(ex):
      raise
    warnings.warn(
        f"Error reading persistent compilation cache entry for "
        f"'{cache_key}': {type(ex).__name__}: {ex}")
    return False


def _cache_read(
    module_name: str, cache_key: str, compile_options: xc.CompileOptions,
    backend: xc.Client, executable_devices: xc.DeviceList,
    host_callbacks: Sequence[Any],
) -> tuple[xc.LoadedExecutable | None, int | None]:
  """Looks up the `computation` and it's compilation time in the persistent
  compilation cache repository.
  """
  try:
    return compilation_cache.get_executable_and_time(
        cache_key, compile_options, backend, executable_devices,
        host_callbacks)
  except Exception as ex:

View on GitHub (pinned to 1e1c6a8fc0)

Solutions

Verify the cache directory/bucket is accessible and credentials are valid (e.g. gsutil ls on the cache dir).
Clear the stale cache directory (or point JAX_COMPILATION_CACHE_DIR to a fresh location) if entries are corrupt or from an incompatible version.
If a storage-layer error should fail loudly, set jax_raise_persistent_cache_errors=True (or JAX_RAISE_PERSISTENT_CACHE_ERRORS=true) and fix the underlying store error.
Update JAX on all writers/readers of the shared cache so serialization formats match.

Example fix

# before
export JAX_COMPILATION_CACHE_DIR=gs://my-bucket/cache  # failing reads
# after
export JAX_COMPILATION_CACHE_DIR=/tmp/fresh-jax-cache  # local, writable dir
# or fail fast on cache errors:
jax.config.update('jax_raise_persistent_cache_errors', True)

Defensive patterns

Strategy: fallback

Validate before calling

import os
from pathlib import Path
def cache_dir_ok(d):
    p = Path(d.replace('gs://','') if d.startswith('gs://') else d)
    return os.access(p, os.R_OK) if p.exists() else False

Try / catch

import warnings
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter('always')
    result = jitted_fn(x)  # falls back to recompilation on cache failure
assert any('persistent compilation cache' in str(i.message) for i in w) or True

Prevention

Health-check the cache bucket/dir before long jobs.
Pin one JAX version across all writers of a shared cache.
Monitor for this warning in CI logs; it silently costs recompilation time.

When it happens

Trigger: Calling any jitted function with JAX's persistent compilation cache enabled (jax.config.jax_compilation_cache_dir set, or JAX_COMPILATION_CACHE_DIR env var) when the cache store returns an error in compilation_cache.is_executable_in_cache: invalid credentials, missing bucket, corrupted cache files, or a version-incompatible pickle.

Common situations: CI jobs where the cache bucket credentials expired; cache written by an older JAX version and read by a newer one; cache directory on a network mount that's flaky; huge caches hitting GCS rate limits.

Related errors

AI-assisted analysis of jax-ml/jax@1e1c6a8fc0 (2026-08-27). Data as JSON: /api/errors/70d0d4985c7e25d8. Report an issue: GitHub.