BerriAI/litellm · error · ValueError

Blocking web crawlers is an enterprise feature. You must be…

Error message

Blocking web crawlers is an enterprise feature. You must be a LiteLLM Enterprise user to use this feature. If you have a license please set `LITELLM_LICENSE` in your env. Get a 7 day trial key here: https://www.litellm.ai/enterprise#trial. 
Pricing: https://www.litellm.ai/#pricing

What it means

Raised by the LiteLLM Proxy when general_settings.block_robots is enabled (true, or a truthy string coerced via str_to_bool) but no valid LiteLLM Enterprise license is present (premium_user is not True). It is a hard startup/config gating error: robot blocking for the proxy UI/docs is an enterprise-only feature. The appended CommonProxyErrors.not_premium_user text tells you to set the LITELLM_LICENSE env var or get a trial key.

Solutions

Get an enterprise trial/license key and set the LITELLM_LICENSE environment variable, then restart the proxy
If you do not have a license, remove `block_robots` from general_settings (the proxy then starts without robot blocking)
Verify the license actually reaches the process: print os.environ.get('LITELLM_LICENSE') in the proxy container / check the premium_user resolution logs
If you have a license and still see this, upgrade litellm and litellm-enterprise to matching versions so license validation succeeds

Example fix

# before (config.yaml)
general_settings:
  block_robots: true   # raises ValueError, no license

# after (option 1: license)
export LITELLM_LICENSE="sk-litellm-enterprise-..."
# config.yaml unchanged

# after (option 2: no license)
general_settings: {}   # block_robots removed

Defensive patterns

Strategy: validation

Validate before calling

import os

has_license = bool(os.environ.get("LITELLM_LICENSE"))
wants_block_robots = True  # from your config build step

if wants_block_robots and not has_license:
    raise SystemExit(
        "block_robots requires LiteLLM Enterprise: set LITELLM_LICENSE or drop the setting"
    )

Prevention

Treat block_robots as an enterprise-only flag in config templates; gate it behind a license check in your config pipeline
Add a CI config-lint that fails if block_robots is set without LITELLM_LICENSE present in the deploy environment
Document which env vars each deployment template needs, and assert required vars at container entrypoint

When it happens

Trigger: Setting `general_settings: block_robots: true` (or "true") in the proxy config.yaml without LITELLM_LICENSE set; passing a block_robots setting while running the OSS proxy; running a container where LITELLM_LICENSE was not propagated; using an expired/invalid license key so premium_user resolves False.

Common situations: Self-hosting the proxy publicly and wanting to hide it from search crawlers without a subscription; license expired after a 7-day trial; docker-compose/k8s secret not mounted so the env var is missing at config load.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/634e3aace9902946. Report an issue: GitHub.

Appendix: source

Thrown at enterprise/litellm_enterprise/proxy/utils.py:31

    general_settings:
      block_robots: true
    ```
    """
    from litellm.proxy.proxy_server import (
        CommonProxyErrors,
        general_settings,
        premium_user,
    )

    _block_robots: Union[bool, str] = general_settings.get("block_robots", False)
    block_robots: Optional[bool] = None
    if isinstance(_block_robots, bool):
        block_robots = _block_robots
    elif isinstance(_block_robots, str):
        block_robots = str_to_bool(_block_robots)
    if block_robots is True:
        if premium_user is not True:
            raise ValueError(
                f"Blocking web crawlers is an enterprise feature. {CommonProxyErrors.not_premium_user.value}"
            )
        return True
    return False

View on GitHub (pinned to 77b7c6c40c)