{"record":{"id":"32d9380518443a3c","repo":"BerriAI/litellm","slug":"mcp-semantic-tool-filtering-could-not-run-embeddi","errorCode":null,"errorMessage":"MCP semantic tool filtering could not run: embedding model '{embedding_model}' exceeded its context window while embedding {stage}. The request was blocked instead of silently passing all tools through. Switch to an embedding model with a larger context window, or disable semantic tool filtering.","messagePattern":"MCP semantic tool filtering could not run: embedding model '(.+?)' exceeded its context window while embedding (.+?)\\. The request was blocked instead of silently passing all tools through\\. Switch to an embedding model with a larger context window, or disable semantic tool filtering\\.","errorType":"exception","errorClass":"SemanticToolFilterContextWindowError","httpStatus":null,"severity":"error","filePath":"litellm/proxy/_experimental/mcp_server/semantic_tool_filter.py","lineNumber":285,"sourceCode":"        Filter tools semantically based on query.\n\n        Args:\n            query: User query to match against tools\n            available_tools: Full list of available MCP tools\n            top_k: Override default top_k (optional)\n\n        Returns:\n            Filtered and ordered list of tools (up to top_k)\n        \"\"\"\n        # Early returns for cases where we can't/shouldn't filter\n        if not self.enabled:\n            return available_tools\n\n        if not available_tools:\n            return available_tools\n\n        if self.context_window_error is not None:\n            raise SemanticToolFilterContextWindowError(\n                embedding_model=self.embedding_model,\n                stage=\"the MCP tool descriptions during semantic router build\",\n                original_error=self.context_window_error,\n            )\n\n        if not query or not query.strip():\n            return available_tools\n\n        # Run semantic filtering\n        try:\n            await self._ensure_tools_indexed(available_tools)\n\n            if self.tool_router is None:\n                verbose_logger.warning(\"Semantic router could not be built from the request's tools\")\n                return available_tools\n\n            available_names: Final = [name for name in (self._extract_tool_info(t)[0] for t in available_tools) if name]\n            if not available_names:","sourceCodeStart":267,"sourceCodeEnd":303,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/proxy/_experimental/mcp_server/semantic_tool_filter.py#L267-L303","documentation":"With litellm_settings.mcp_semantic_tool_filter.enabled: true, LiteLLM builds a semantic index over all registered MCP tool descriptions. If that build fails because the embedding model's context window was exceeded, the failure is stored on the filter; every later filter call re-raises SemanticToolFilterContextWindowError instead of silently passing all tools through. This is a deliberate fail-closed design: the request is blocked rather than risking an oversized tool payload.","triggerScenarios":"Many MCP servers registered so the concatenated tool descriptions exceed the embedding model's token limit (8191 for text-embedding-3-*); the startup build records the overflow in self.context_window_error, and each chat completion using MCP tools then raises this error from the early-return check.","commonSituations":"Onboarding dozens of MCP servers with verbose tool descriptions while keeping the default embedding model; catalogs that grew past the token ceiling after new servers were added.","solutions":["Configure a larger-context embedding model under litellm_settings.mcp_semantic_tool_filter.embedding_model (e.g., voyage-3 with a 32k window).","Reduce indexed content: unregister unused MCP servers or shorten their tool descriptions.","Disable the filter — litellm_settings.mcp_semantic_tool_filter.enabled: false — to fall back to passing tools through unfiltered (then watch the LLM's own context budget).","Restart/rebuild the semantic router after fixing either the model or the catalog."],"exampleFix":"# before\nlitellm_settings:\n  mcp_semantic_tool_filter:\n    enabled: true\n    # default embedding model context window too small for the catalog\n\n# after\nlitellm_settings:\n  mcp_semantic_tool_filter:\n    enabled: true\n    embedding_model: voyage/voyage-3","handlingStrategy":"fallback","validationCode":"def approx_index_tokens(tools: list[dict]) -> int:\n    return sum(len(t.get(\"description\") or \"\") + len(t.get(\"name\") or \"\") for t in tools) // 4\n\n# before enabling the filter, make sure the catalog fits the embedding window\nif approx_index_tokens(all_tools) > 7000:  # safety margin under an 8191-token window\n    disable_semantic_filter_or_switch_model()","typeGuard":null,"tryCatchPattern":"try:\n    resp = await client.post(f\"{base}/v1/chat/completions\", json=payload)\n    resp.raise_for_status()\nexcept (httpx.HTTPStatusError, Exception) as e:\n    if \"exceeded its context window\" in str(e) and \"semantic tool filtering\" in str(e):\n        # fail-closed by design: fix config (bigger-window embedding model or\n        # enabled: false), then replay the request; do not blind-retry\n        raise ToolCatalogTooLarge(str(e)) from e\n    raise","preventionTips":["Estimate total tool-description tokens before enabling semantic filtering; keep a margin under the embedding window.","Trim tool descriptions and unregister unused MCP servers as the catalog grows.","Monitor the x-litellm-semantic-filter response headers to catch filter degradation early."],"tags":["mcp","embeddings","semantic-search","context-window","configuration"],"backgroundTag":"embedding-context-window-exceeded","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}