BerriAI/litellm · error · ValueError

target_model_names is required for this routing scenario

Error message

target_model_names is required for this routing scenario

What it means

In GET /v1/batches, SCENARIO 2 handles target_model_names routing (query param or JSON body). The code re-assigns target_model_names from whichever source is set, then raises ValueError('target_model_names is required for this routing scenario') if it is still None. Because the enclosing elif only runs when one of the two sources is truthy, this is a defensive invariant check whose failure branch normal requests cannot reach; if it fires, the ValueError escapes as an unhandled server error (HTTP 500), signalling a code regression or mutated request state rather than bad caller input.

Source

Thrown at litellm/proxy/batches_endpoints/endpoints.py:792

                after=after,
                limit=limit,
                **data,
            )

            # Encode batch IDs in the list response so clients can use
            # them for retrieve/cancel/file downloads through the proxy.
            response_data: Final = getattr(response, "data", None)
            if response_data:
                for batch in response_data:
                    encode_batch_response_ids(batch, model=model_param)

            verbose_proxy_logger.debug("Listed batches using model: %s", model_param)

        # SCENARIO 2 (alternative): target_model_names based routing
        elif target_model_names or data.get("target_model_names", None):
            target_model_names = target_model_names or data.get("target_model_names", None)
            if target_model_names is None:
                raise ValueError("target_model_names is required for this routing scenario")
            model: Final = target_model_names.split(",")[0]
            data.pop("model", None)
            response = await llm_router.alist_batches(
                model=model,
                after=after,
                limit=limit,
                **data,
            )

        # SCENARIO 3: Fallback to custom_llm_provider (uses env variables)
        else:
            custom_llm_provider: Final = (
                provider
                or get_custom_llm_provider_from_request_headers(request=request)
                or get_custom_llm_provider_from_request_query(request=request)
                or "openai"
            )
            apply_team_provider_credentials(

View on GitHub (pinned to 77b7c6c40c)

Solutions

Upgrade to the latest litellm; this guard path may have been reworked.
Capture the traceback plus the exact request (query string and body) and open an issue at github.com/BerriAI/litellm.
Workaround: send target_model_names as a query parameter only, avoiding duplicate body keys.

Defensive patterns

Strategy: try-catch

Try / catch

try:
    page = client.batches.list(extra_query={'target_model_names': name})
except openai.InternalServerError as e:
    if 'target_model_names is required for this routing scenario' in str(e):
        # proxy-side invariant trip - not an input problem; do not retry blindly
        capture_request_for_bug_report()
        raise

Prevention

Pass target_model_names as a single query parameter instead of duplicating it in the JSON body.
Keep litellm versions uniform across environments.
Report reproducible occurrences with the full traceback to the litellm maintainers.

When it happens

Trigger: Not reachable through well-formed requests - the elif condition guarantees a non-None value; it would require the request-body dict to change between check and assignment, or a modified/older build of this routing code.

Common situations: Encountered only as an unexpected 500 whose traceback contains this ValueError, usually on patched or older LiteLLM versions where the branch structure differs.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/9303a6bd345dee90. Report an issue: GitHub.