{"record":{"id":"8ac48f7bea8b55d3","repo":"BerriAI/litellm","slug":"update-eval-is-not-supported-for-custom-llm-provi","errorCode":null,"errorMessage":"UPDATE eval is not supported for {custom_llm_provider}","messagePattern":"UPDATE eval is not supported for (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/evals/main.py","lineNumber":679,"sourceCode":"    try:\n        litellm_logging_obj: Final[LiteLLMLoggingObj] = kwargs.get(\"litellm_logging_obj\")\n        litellm_call_id: Final[str | None] = kwargs.get(\"litellm_call_id\", None)\n        _is_async: Final = kwargs.pop(\"aupdate_eval\", False) is True\n\n        # Get LiteLLM parameters\n        litellm_params: Final = GenericLiteLLMParams(**kwargs)\n\n        # Determine provider\n        if custom_llm_provider is None:\n            custom_llm_provider = \"openai\"\n\n        # Get provider config\n        evals_api_provider_config: BaseEvalsAPIConfig | None = ProviderConfigManager.get_provider_evals_api_config(\n            provider=litellm.LlmProviders(custom_llm_provider),\n        )\n\n        if evals_api_provider_config is None:\n            raise ValueError(f\"UPDATE eval is not supported for {custom_llm_provider}\")\n\n        # Build update request\n        update_request: Final[UpdateEvalRequest] = {}\n        if name is not None:\n            update_request[\"name\"] = name\n\n        # Filter metadata to exclude internal LiteLLM fields\n        if metadata is not None:\n            # List of internal LiteLLM metadata keys that should NOT be sent to OpenAI\n            internal_keys: Final = {\n                \"headers\",\n                \"requester_metadata\",\n                \"user_api_key_hash\",\n                \"user_api_key_alias\",\n                \"user_api_key_spend\",\n                \"user_api_key_max_budget\",\n                \"user_api_key_team_id\",\n                \"user_api_key_user_id\",","sourceCodeStart":661,"sourceCodeEnd":697,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/evals/main.py#L661-L697","documentation":"Raised in litellm.evals.update_eval (litellm/evals/main.py:679) when the resolved provider lacks an Evals API config. ProviderConfigManager.get_provider_evals_api_config supports only LlmProviders.OPENAI, so update operations against any other provider are rejected before the update_request dict is built.","triggerScenarios":"Calling litellm.evals.update_eval(eval_id=..., custom_llm_provider=<non-openai>); attempting to rename or update testing criteria on an eval while pointed at Azure/Anthropic/Bedrock.","commonSituations":"Multi-provider deployments reusing a provider variable; assuming evals CRUD works everywhere because completion() does.","solutions":["Pass custom_llm_provider='openai' (default) with a valid OPENAI_API_KEY","Use the provider's native REST API for eval updates outside OpenAI","Guard eval-management features in your app to OpenAI-only until LiteLLM adds configs"],"exampleFix":"# before\nlitellm.evals.update_eval(eval_id=\"eval_123\", name=\"v2\", custom_llm_provider=\"anthropic\")\n\n# after\nlitellm.evals.update_eval(eval_id=\"eval_123\", name=\"v2\", custom_llm_provider=\"openai\")","handlingStrategy":"validation","validationCode":"if custom_llm_provider != \"openai\":\n    raise ValueError(\"update_eval requires the OpenAI provider\")\nlitellm.evals.update_eval(eval_id=eval_id, custom_llm_provider=\"openai\", ...)","typeGuard":"def evals_provider_supported(provider: str) -> bool:\n    return provider == \"openai\"","tryCatchPattern":"try:\n    litellm.evals.update_eval(eval_id=eval_id, name=name)\nexcept ValueError as e:\n    if \"UPDATE eval is not supported\" in str(e):\n        logger.error(\"switch evals calls to custom_llm_provider='openai'\")\n        raise","preventionTips":["Pin the evals provider constant to 'openai' in config","Gate eval-management UI features on provider == openai","Re-check supported providers after each litellm upgrade"],"tags":["evals","provider-support","openai","validation"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}