{"record":{"id":"b5f643f8f6671d62","repo":"BerriAI/litellm","slug":"internal-error-cache-cannot-be-empty-internal-u","errorCode":null,"errorMessage":"Internal Error: Cache cannot be empty - internal_usage_cache={internal_usage_cache}","messagePattern":"Internal Error: Cache cannot be empty - internal_usage_cache=(.+?)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"litellm/litellm_core_utils/litellm_logging.py","lineNumber":4141,"sourceCode":"            )\n            for callback in _in_memory_loggers:\n                # Use exact type check to avoid matching ArizePhoenixLogger (subclass)\n                if type(callback) is OpenTelemetry:\n                    return callback\n            _otel_logger = OpenTelemetry(config=otel_config)\n            _in_memory_loggers.append(_otel_logger)\n            return _otel_logger\n        elif logging_integration == \"dynamic_rate_limiter\":\n            from litellm.proxy.hooks.dynamic_rate_limiter import (\n                _PROXY_DynamicRateLimitHandler,\n            )\n\n            for callback in _in_memory_loggers:\n                if isinstance(callback, _PROXY_DynamicRateLimitHandler):\n                    return callback\n\n            if internal_usage_cache is None:\n                raise Exception(f\"Internal Error: Cache cannot be empty - internal_usage_cache={internal_usage_cache}\")\n\n            dynamic_rate_limiter_obj: Final = _PROXY_DynamicRateLimitHandler(internal_usage_cache=internal_usage_cache)\n\n            if llm_router is not None and isinstance(llm_router, litellm.Router):\n                dynamic_rate_limiter_obj.update_variables(llm_router=llm_router)\n            _in_memory_loggers.append(dynamic_rate_limiter_obj)\n            return dynamic_rate_limiter_obj\n        elif logging_integration == \"dynamic_rate_limiter_v3\":\n            from litellm.proxy.hooks.dynamic_rate_limiter_v3 import (\n                _PROXY_DynamicRateLimitHandlerV3,\n            )\n\n            for callback in _in_memory_loggers:\n                if isinstance(callback, _PROXY_DynamicRateLimitHandlerV3):\n                    return callback\n\n            if internal_usage_cache is None:\n                raise Exception(f\"Internal Error: Cache cannot be empty - internal_usage_cache={internal_usage_cache}\")","sourceCodeStart":4123,"sourceCodeEnd":4159,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/litellm_core_utils/litellm_logging.py#L4123-L4159","documentation":"Raised when the 'dynamic_rate_limiter' proxy callback is instantiated without an internal usage cache. _PROXY_DynamicRateLimitHandler adjusts limits from observed TPM/RPM usage stored in LiteLLM's internal_usage_cache; the logging initializer passes that cache only when running inside a properly initialized LiteLLM proxy, so SDK-side use (or a proxy state where the cache was never created) makes internal_usage_cache None and this Exception fires.","triggerScenarios":"Setting litellm.callbacks = ['dynamic_rate_limiter'] in plain SDK code (no proxy); enabling the callback before proxy cache initialization; running an older/newer proxy where internal_usage_cache is not wired into initialize's arguments.","commonSituations":"Reusing proxy-style config (litellm_settings.callbacks) in SDK scripts; minimal proxy deployments without any cache configured (no litellm_settings.cache block / redis_url); version drift between litellm core and proxy extras.","solutions":["Use dynamic_rate_limiter only inside the LiteLLM proxy with a cache configured (litellm_settings: cache: true, ideally redis_url for multi-instance).","Upgrade litellm to a matching current version so the proxy passes internal_usage_cache.","Remove the callback if you do not actually need usage-based dynamic rate limiting."],"exampleFix":"# before (SDK)\nimport litellm\nlitellm.callbacks = ['dynamic_rate_limiter']  # Exception: Internal Error: Cache cannot be empty\n\n# after: run it on the proxy with a cache\n# config.yaml\n# litellm_settings:\n#   cache: true\n#   callbacks: ['dynamic_rate_limiter']","handlingStrategy":"validation","validationCode":"# dynamic_rate_limiter is proxy-only: verify you are on the proxy with a cache before enabling it\nimport litellm\ncache_ok = getattr(litellm, 'cache', None) is not None or os.environ.get('REDIS_HOST')\nif not cache_ok:\n    raise RuntimeError('enable litellm_settings.cache (and redis) before using dynamic_rate_limiter')","typeGuard":null,"tryCatchPattern":"try:\n    litellm.completion(model='gpt-4o', messages=m)\nexcept Exception as e:\n    if 'Internal Error: Cache cannot be empty' in str(e):\n        # remove 'dynamic_rate_limiter' from callbacks or configure litellm_settings.cache on the proxy\n        raise\n    raise","preventionTips":["Treat dynamic_rate_limiter(_v3) as a proxy feature: never add it to plain SDK callbacks.","Config lint: if callbacks include dynamic_rate_limiter, require a cache block in config.yaml."],"tags":["litellm","rate-limiting","proxy","callbacks","cache"],"backgroundTag":"cache-not-configured","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}