{"record":{"id":"634e3aace9902946","repo":"BerriAI/litellm","slug":"blocking-web-crawlers-is-an-enterprise-feature-yo","errorCode":null,"errorMessage":"Blocking web crawlers is an enterprise feature. You must be a LiteLLM Enterprise user to use this feature. If you have a license please set `LITELLM_LICENSE` in your env. Get a 7 day trial key here: https://www.litellm.ai/enterprise#trial. \nPricing: https://www.litellm.ai/#pricing","messagePattern":"Blocking web crawlers is an enterprise feature\\. You must be a LiteLLM Enterprise user to use this feature\\. If you have a license please set `LITELLM_LICENSE` in your env\\. Get a 7 day trial key here: https://www\\.litellm\\.ai/enterprise#trial\\. \nPricing: https://www\\.litellm\\.ai/#pricing","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"enterprise/litellm_enterprise/proxy/utils.py","lineNumber":31,"sourceCode":"    general_settings:\n      block_robots: true\n    ```\n    \"\"\"\n    from litellm.proxy.proxy_server import (\n        CommonProxyErrors,\n        general_settings,\n        premium_user,\n    )\n\n    _block_robots: Union[bool, str] = general_settings.get(\"block_robots\", False)\n    block_robots: Optional[bool] = None\n    if isinstance(_block_robots, bool):\n        block_robots = _block_robots\n    elif isinstance(_block_robots, str):\n        block_robots = str_to_bool(_block_robots)\n    if block_robots is True:\n        if premium_user is not True:\n            raise ValueError(\n                f\"Blocking web crawlers is an enterprise feature. {CommonProxyErrors.not_premium_user.value}\"\n            )\n        return True\n    return False\n","sourceCodeStart":13,"sourceCodeEnd":36,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/enterprise/litellm_enterprise/proxy/utils.py#L13-L36","documentation":"A startup-time ValueError from enterprise utils: general_settings contains block_robots=true, which is an Enterprise-only feature, and premium_user is not True (no valid LITELLM_LICENSE). The proxy validates this setting during config load and aborts rather than silently running the non-enterprise behavior.","triggerScenarios":"Enabling 'block_robots': true in general_settings of the proxy config while LITELLM_LICENSE is unset, expired, or invalid. The check `block_robots is True and premium_user is not True` fires and raises.","commonSituations":"Copying an enterprise example config into a community deployment; trial license expiring between restarts; LITELLM_LICENSE env var not passed into the container so the premium check fails after a redeploy.","solutions":["Set a valid LITELLM_LICENSE (or trial key from https://www.litellm.ai/enterprise#trial) in the proxy environment and restart.","If you do not intend to use the feature, remove 'block_robots' (or set it to false) from general_settings in the config.","Verify the license env var actually reaches the process (docker -e, k8s secret, systemd Environment=) and check startup logs for license validation.","Pin a known-good deployment after fixing, since this error prevents the proxy from starting."],"exampleFix":"# before (config.yaml)\ngeneral_settings:\n  block_robots: true  # ValueError: enterprise feature, no license\n\n# after: either license it...\n# export LITELLM_LICENSE='<your-enterprise-key>'\n\n# ...or drop the setting\ngeneral_settings: {}","handlingStrategy":"validation","validationCode":"import os, yaml\ncfg = yaml.safe_load(open('config.yaml'))\nif cfg.get('general_settings', {}).get('block_robots'):\n    if not os.environ.get('LITELLM_LICENSE'):\n        raise RuntimeError('block_robots requires LITELLM_LICENSE')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate config against license state in CI before deploy: enterprise settings require LITELLM_LICENSE present.","Keep community-only configs free of enterprise settings like block_robots.","Monitor license expiry — this ValueError blocks proxy startup when it lapses."],"tags":["litellm","enterprise","licensing","configuration","startup"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}