{"record":{"id":"c8db64ff1a416964","repo":"ZhuLinsen/daily_stock_analysis","slug":"hermes-non-hermes-mixed-generation-route-is-not-su","errorCode":null,"errorMessage":"Hermes/non-Hermes mixed generation route is not supported without deployment-level no-proxy client support","messagePattern":"Hermes/non-Hermes mixed generation route is not supported without deployment-level no-proxy client support","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"src/analyzer.py","lineNumber":2757,"sourceCode":"    ) -> str:\n        runtime_config = config or self._get_runtime_config()\n        redactions = self._litellm_redaction_values_for_model(runtime_config, model)\n        sanitized = sanitize_hermes_error_text(exc, redaction_values=redactions)\n        return redact_diagnostic_text(sanitized, limit=500)\n\n    def _dispatch_litellm_completion(\n        self,\n        model: str,\n        call_kwargs: Dict[str, Any],\n        *,\n        config: Config,\n        use_channel_router: bool,\n        router_model_names: set[str],\n    ) -> Any:\n        \"\"\"Dispatch a LiteLLM completion through router or direct fallback.\"\"\"\n        origins = route_deployment_origins(config.llm_model_list, model)\n        if origins.is_mixed:\n            raise RuntimeError(\"Hermes/non-Hermes mixed generation route is not supported without deployment-level no-proxy client support\")\n        if origins.is_hermes_only:\n            deployment = origins.hermes_deployments[0]\n            params = dict(deployment.get(\"litellm_params\") or {})\n            api_key = str(params.get(\"api_key\") or \"\").strip()\n            base_url = str(params.get(\"api_base\") or \"\").strip()\n            if is_masked_secret_placeholder(api_key):\n                raise RuntimeError(\"Hermes API key is a masked placeholder and cannot be used for generation\")\n            timeout = float(call_kwargs.get(\"timeout\") or 30.0)\n            hermes_kwargs = dict(call_kwargs)\n            hermes_kwargs[\"model\"] = str(params.get(\"model\") or model)\n            hermes_kwargs[\"stream\"] = False\n            hermes_kwargs.pop(\"api_key\", None)\n            hermes_kwargs.pop(\"api_base\", None)\n            with open_hermes_no_proxy_client(api_key=api_key, base_url=base_url, timeout=timeout) as client:\n                hermes_kwargs[\"client\"] = client\n                return litellm.completion(**hermes_kwargs)\n\n        wire_models = resolve_fallback_litellm_wire_models(model, config.llm_model_list)","sourceCodeStart":2739,"sourceCodeEnd":2775,"githubUrl":"https://github.com/ZhuLinsen/daily_stock_analysis/blob/5159bd72e8373d215492dff122acc9d389e219c9/src/analyzer.py#L2739-L2775","documentation":"RuntimeError raised in _dispatch_litellm_completion when route_deployment_origins reports that the requested model's LiteLLM deployment list mixes Hermes-origin and non-Hermes deployments. Hermes deployments need a dedicated no-proxy HTTP client (open_hermes_no_proxy_client), which only works when ALL traffic for that model goes through the direct Hermes path — a mixed router cannot honor both client regimes, so generation aborts before calling LiteLLM.","triggerScenarios":"Calling LLM generation with config.llm_model_list containing a model name mapped to 2+ deployments where at least one is Hermes-tagged and at least one is not (e.g. one deployment behind a proxy and one Hermes direct). The check fires per model at dispatch time, regardless of whether the mixed fallback would ever be exercised.","commonSituations":"Adding a Hermes deployment as an extra fallback under an existing model alias; merging LLM_MODEL_LIST configs from two environments; a proxy-shared config where some deployments carry Hermes markers and others do not.","solutions":["Split the mixed deployments into separate model names (e.g. 'gpt4-hermes' vs 'gpt4-proxy') so each alias routes to a single origin kind.","Or make the model's deployment list uniformly Hermes (or uniformly non-Hermes).","After editing LLM_MODEL_LIST, verify with route_deployment_origins(config.llm_model_list, model) that is_mixed is False for every model you will use."],"exampleFix":"# before (config: model 'main' -> [hermes_deployment, proxied_deployment])\n# RuntimeError: Hermes/non-Hermes mixed generation route ...\n\n# after (config: 'main' -> [proxied_deployment], 'main-hermes' -> [hermes_deployment])\nresponse = dispatch_litellm_completion(\"main-hermes\", kwargs)","handlingStrategy":"validation","validationCode":"from src.analyzer import route_deployment_origins  # or its defining module\n\norigins = route_deployment_origins(config.llm_model_list, model_name)\nif origins.is_mixed:\n    raise ConfigError(f\"model {model_name!r} mixes Hermes and non-Hermes deployments; split them\")","typeGuard":null,"tryCatchPattern":"try:\n    result = analyzer._dispatch_litellm_completion(...)\nexcept RuntimeError as e:\n    if \"mixed generation route\" in str(e):\n        # config-level bug: fix llm_model_list, do not retry\n        raise ConfigError(str(e)) from e\n    raise","preventionTips":["One origin kind per model alias; use separate aliases for Hermes and proxied deployments.","Validate llm_model_list with route_deployment_origins at startup for every model in use.","Review merged configs from other environments for deployment origin tags."],"tags":["llm","litellm","hermes","routing","configuration"],"backgroundTag":null,"analyzedSha":"5159bd72e8373d215492dff122acc9d389e219c9","analyzedAt":"2026-08-15T01:59:36.292Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}