BerriAI/litellm · error · ValueError

Prometheus exclude configuration validation failed

Error message

Prometheus exclude configuration validation failed:
{joined_errors}

What it means

The global exclude settings — prometheus_exclude_metrics and prometheus_exclude_labels — are validated against closed sets: metric names must be among DEFINED_PROMETHEUS_METRICS (Literal args) and label names among every label a metric can emit (enum labels, hard-coded labels, sanitized custom labels). Any unknown name yields one line per offender, and all lines are joined into this ValueError.

Solutions

1. Correct each name listed verbatim in the error text
2. Enumerate valid metric names from litellm.integrations.prometheus.DIFF... (DEFINED_PROMETHEUS_METRICS Literal args, e.g. via typing.get_args) and valid labels via the callback's _all_defined_labels()
3. Drop stale exclude entries after upgrading litellm
4. Keep exclude lists in version control next to the litellm version pin so they drift visibly

Example fix

# before
os.environ['PROMETHEUS_EXCLUDE_METRICS'] = 'litellm_llm_api_latency_metric,typo_metric'

# after
from typing import get_args
from litellm.integrations.prometheus import DEFINED_PROMETHEUS_METRICS
valid = set(get_args(DEFINED_PROMETHEUS_METRICS))
excludes = [m for m in ['litellm_llm_api_latency_metric', 'typo_metric'] if m in valid]
os.environ['PROMETHEUS_EXCLUDE_METRICS'] = ','.join(excludes)

from typing import get_args
from litellm.integrations.prometheus import DEFINED_PROMETHEUS_METRICS

valid_metrics = set(get_args(DEFINED_PROMETHEUS_METRICS))
invalid = [m for m in exclude_metrics_list if m not in valid_metrics]
assert not invalid, f'unknown metrics in exclude list: {invalid}'

When it happens

Trigger: Setting prometheus_exclude_metrics=['litellm_proxy_requst_total'] (typo) or excluding a label that no metric defines; excludes written for an older litellm version whose metric/label inventory changed; excluding a custom label while custom_prometheus_metadata_labels is empty.

Common situations: Version drift after upgrading litellm (renamed metrics), copy-pasted exclude lists between projects, excludes for labels that only exist when certain callbacks or tag configs are active.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Configuration validation failed
• {error.message}
• {label_error.message}
• bucket_name must be provided for GCS destination
• Custom code guardrail requires a guardrail_name

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/05bd8d3ca0ddd173. Report an issue: GitHub.

Appendix: source

Thrown at litellm/integrations/prometheus.py:738

        from typing import get_args

        import litellm

        exclude_metrics: Final = frozenset(litellm.prometheus_exclude_metrics or ())
        exclude_labels: Final = frozenset(litellm.prometheus_exclude_labels or ())

        valid_metrics: Final = frozenset(get_args(DEFINED_PROMETHEUS_METRICS))
        invalid_metrics: Final = sorted(exclude_metrics - valid_metrics)

        valid_labels: Final = self._all_defined_labels()
        invalid_labels: Final = sorted(exclude_labels - valid_labels)

        errors: Final = (
            *(f"Invalid metric name in prometheus_exclude_metrics: {metric}" for metric in invalid_metrics),
            *(f"Invalid label name in prometheus_exclude_labels: {label}" for label in invalid_labels),
        )
        if errors:
            raise ValueError("Prometheus exclude configuration validation failed:\n" + "\n".join(errors))

        return exclude_metrics, exclude_labels

    @staticmethod
    def _all_defined_labels() -> frozenset[str]:
        """Every label a metric can emit: enum labels, hard-coded labels, and configured custom labels / tags."""
        import litellm

        builtin_labels: Final = frozenset(label.value for label in UserAPIKeyLabelNames)
        custom_metadata_labels: Final = frozenset(
            _sanitize_prometheus_label_name(label) for label in litellm.custom_prometheus_metadata_labels
        )
        custom_tag_labels: Final = frozenset(
            _sanitize_prometheus_label_name(f"tag_{tag}") for tag in litellm.custom_prometheus_tags
        )
        return builtin_labels | _NON_ENUM_METRIC_LABELS | custom_metadata_labels | custom_tag_labels

    def _validate_all_configurations(self, parsed_configs: list) -> ValidationResults:

View on GitHub (pinned to 77b7c6c40c)