BerriAI/litellm · error · Exception

Internal Error: Cache cannot be empty - internal_usage_cache

Error message

Internal Error: Cache cannot be empty - internal_usage_cache={internal_usage_cache}

What it means

The 'dynamic_rate_limiter' callback is a proxy hook that reads project/team usage from the proxy's shared Redis-backed internal usage cache; it cannot function without one. When LiteLLM lazily instantiates _PROXY_DynamicRateLimitHandler and internal_usage_cache is None (no proxy cache configured), it raises this Exception. The 'Internal Error' prefix signals a server-side configuration gap rather than a request problem.

Source

Thrown at litellm/litellm_core_utils/litellm_logging.py:4135

            )
            for callback in _in_memory_loggers:
                # Use exact type check to avoid matching ArizePhoenixLogger (subclass)
                if type(callback) is OpenTelemetry:
                    return callback
            _otel_logger = OpenTelemetry(config=otel_config)
            _in_memory_loggers.append(_otel_logger)
            return _otel_logger
        elif logging_integration == "dynamic_rate_limiter":
            from litellm.proxy.hooks.dynamic_rate_limiter import (
                _PROXY_DynamicRateLimitHandler,
            )

            for callback in _in_memory_loggers:
                if isinstance(callback, _PROXY_DynamicRateLimitHandler):
                    return callback

            if internal_usage_cache is None:
                raise Exception(f"Internal Error: Cache cannot be empty - internal_usage_cache={internal_usage_cache}")

            dynamic_rate_limiter_obj: Final = _PROXY_DynamicRateLimitHandler(internal_usage_cache=internal_usage_cache)

            if llm_router is not None and isinstance(llm_router, litellm.Router):
                dynamic_rate_limiter_obj.update_variables(llm_router=llm_router)
            _in_memory_loggers.append(dynamic_rate_limiter_obj)
            return dynamic_rate_limiter_obj
        elif logging_integration == "dynamic_rate_limiter_v3":
            from litellm.proxy.hooks.dynamic_rate_limiter_v3 import (
                _PROXY_DynamicRateLimitHandlerV3,
            )

            for callback in _in_memory_loggers:
                if isinstance(callback, _PROXY_DynamicRateLimitHandlerV3):
                    return callback

            if internal_usage_cache is None:
                raise Exception(f"Internal Error: Cache cannot be empty - internal_usage_cache={internal_usage_cache}")

View on GitHub (pinned to 6c2dcb801b)

Solutions

Configure a proxy cache in config.yaml (cache: type: redis, url / redis env vars) so the internal usage cache exists, then restart
Or remove 'dynamic_rate_limiter' from callbacks if you don't intend to use usage-based dynamic rate limiting
Verify Redis connectivity from the proxy container (redis-cli ping) since a failed cache init also leaves internal_usage_cache None
Check proxy startup logs for cache initialization errors preceding this one

Example fix

# before
litellm_settings:
  callbacks: ["dynamic_rate_limiter"]   # Exception: Cache cannot be empty

# after
litellm_settings:
  callbacks: ["dynamic_rate_limiter"]
cache:
  type: redis
  url: redis://redis:6379

Defensive patterns

Strategy: validation

Validate before calling

# config-level guard: only enable dynamic rate limiting when a cache is configured
import yaml
cfg = yaml.safe_load(open('config.yaml'))
has_cache = 'cache' in cfg and cfg['cache'].get('type') == 'redis'
callbacks = cfg.get('litellm_settings', {}).get('callbacks', [])
assert not ('dynamic_rate_limiter' in callbacks and not has_cache), \
    'dynamic_rate_limiter requires a Redis cache block in config.yaml'

Prevention

Pair dynamic_rate_limiter with a cache: redis block in the same config
Monitor proxy startup logs for cache initialization failures
Don't enable proxy-only dynamic rate limiter callbacks when using litellm as a library

When it happens

Trigger: Adding 'dynamic_rate_limiter' to litellm_settings.callbacks (or default callbacks) in a LiteLLM proxy config.yaml that has no cache: block (no Redis URL / internal_usage_cache host), then triggering the first logged request.

Common situations: Copy-pasting a config that includes the dynamic rate limiter without the Redis section it was written against; running the proxy locally without Redis; Redis env vars present but the cache: block missing so the internal usage cache is never constructed.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/b5f643f8f6671d62. Report an issue: GitHub.