ZhuLinsen/daily_stock_analysis · critical · RuntimeError

Hermes/non-Hermes mixed generation route is not supported wi

Error message

Hermes/non-Hermes mixed generation route is not supported without deployment-level no-proxy client support

What it means

RuntimeError raised in _dispatch_litellm_completion when route_deployment_origins reports that the requested model's LiteLLM deployment list mixes Hermes-origin and non-Hermes deployments. Hermes deployments need a dedicated no-proxy HTTP client (open_hermes_no_proxy_client), which only works when ALL traffic for that model goes through the direct Hermes path — a mixed router cannot honor both client regimes, so generation aborts before calling LiteLLM.

Source

Thrown at src/analyzer.py:2757

    ) -> str:
        runtime_config = config or self._get_runtime_config()
        redactions = self._litellm_redaction_values_for_model(runtime_config, model)
        sanitized = sanitize_hermes_error_text(exc, redaction_values=redactions)
        return redact_diagnostic_text(sanitized, limit=500)

    def _dispatch_litellm_completion(
        self,
        model: str,
        call_kwargs: Dict[str, Any],
        *,
        config: Config,
        use_channel_router: bool,
        router_model_names: set[str],
    ) -> Any:
        """Dispatch a LiteLLM completion through router or direct fallback."""
        origins = route_deployment_origins(config.llm_model_list, model)
        if origins.is_mixed:
            raise RuntimeError("Hermes/non-Hermes mixed generation route is not supported without deployment-level no-proxy client support")
        if origins.is_hermes_only:
            deployment = origins.hermes_deployments[0]
            params = dict(deployment.get("litellm_params") or {})
            api_key = str(params.get("api_key") or "").strip()
            base_url = str(params.get("api_base") or "").strip()
            if is_masked_secret_placeholder(api_key):
                raise RuntimeError("Hermes API key is a masked placeholder and cannot be used for generation")
            timeout = float(call_kwargs.get("timeout") or 30.0)
            hermes_kwargs = dict(call_kwargs)
            hermes_kwargs["model"] = str(params.get("model") or model)
            hermes_kwargs["stream"] = False
            hermes_kwargs.pop("api_key", None)
            hermes_kwargs.pop("api_base", None)
            with open_hermes_no_proxy_client(api_key=api_key, base_url=base_url, timeout=timeout) as client:
                hermes_kwargs["client"] = client
                return litellm.completion(**hermes_kwargs)

        wire_models = resolve_fallback_litellm_wire_models(model, config.llm_model_list)

View on GitHub (pinned to 5159bd72e8)

Solutions

Split the mixed deployments into separate model names (e.g. 'gpt4-hermes' vs 'gpt4-proxy') so each alias routes to a single origin kind.
Or make the model's deployment list uniformly Hermes (or uniformly non-Hermes).
After editing LLM_MODEL_LIST, verify with route_deployment_origins(config.llm_model_list, model) that is_mixed is False for every model you will use.

Example fix

# before (config: model 'main' -> [hermes_deployment, proxied_deployment])
# RuntimeError: Hermes/non-Hermes mixed generation route ...

# after (config: 'main' -> [proxied_deployment], 'main-hermes' -> [hermes_deployment])
response = dispatch_litellm_completion("main-hermes", kwargs)

Defensive patterns

Strategy: validation

Validate before calling

from src.analyzer import route_deployment_origins  # or its defining module

origins = route_deployment_origins(config.llm_model_list, model_name)
if origins.is_mixed:
    raise ConfigError(f"model {model_name!r} mixes Hermes and non-Hermes deployments; split them")

Try / catch

try:
    result = analyzer._dispatch_litellm_completion(...)
except RuntimeError as e:
    if "mixed generation route" in str(e):
        # config-level bug: fix llm_model_list, do not retry
        raise ConfigError(str(e)) from e
    raise

Prevention

One origin kind per model alias; use separate aliases for Hermes and proxied deployments.
Validate llm_model_list with route_deployment_origins at startup for every model in use.
Review merged configs from other environments for deployment origin tags.

When it happens

Trigger: Calling LLM generation with config.llm_model_list containing a model name mapped to 2+ deployments where at least one is Hermes-tagged and at least one is not (e.g. one deployment behind a proxy and one Hermes direct). The check fires per model at dispatch time, regardless of whether the mixed fallback would ever be exercised.

Common situations: Adding a Hermes deployment as an extra fallback under an existing model alias; merging LLM_MODEL_LIST configs from two environments; a proxy-shared config where some deployments carry Hermes markers and others do not.

Related errors

AI-assisted analysis of ZhuLinsen/daily_stock_analysis@5159bd72e8 (2026-08-15). Data as JSON: /api/errors/c8db64ff1a416964. Report an issue: GitHub.