BerriAI/litellm · error · Exception

Failed to generate embedding for query: {e}

Error message

Failed to generate embedding for query: {e}

What it means

This is a wrapper exception: the search transform calls litellm.embedding(model=embedding_model, input=[query], **embedding_config), and any failure inside that embedding call (auth, bad deployment name, wrong api_version, network) is caught and re-raised as 'Failed to generate embedding for query: <original error>'. The root cause is in the appended message.

Source

Thrown at litellm/llms/azure_ai/vector_stores/transformation.py:158

                "Example: litellm_params['embedding_config'] = {'api_base': 'https://krris-mh44uf7y-eastus2.cognitiveservices.azure.com/', 'api_key': 'os.environ/AZURE_API_KEY', 'api_version': '2025-09-01'}"
            )

        # Get vector field name (defaults to contentVector)
        vector_field: Final = litellm_params.get("azure_search_vector_field", "contentVector")

        # Get top_k (number of results to return)
        top_k: Final = vector_store_search_optional_params.get("top_k", 10)

        # Generate embedding for the query using litellm.embeddings
        try:
            embedding_response: Final = litellm.embedding(
                model=embedding_model,
                input=[query],
                **embedding_config,
            )
            query_vector: Final = embedding_response.data[0]["embedding"]
        except Exception as e:
            raise Exception(f"Failed to generate embedding for query: {e}")

        # Azure AI Search endpoint for search
        index_name: Final = vector_store_id  # vector_store_id is the index name
        url: Final = f"{api_base}/indexes/{index_name}/docs/search?api-version=2024-07-01"

        # Build the request body for Azure AI Search with vector search
        request_body: Final = {
            "search": "*",  # Get all documents (filtered by vector similarity)
            "vectorQueries": [
                {
                    "vector": query_vector,
                    "fields": vector_field,
                    "kind": "vector",
                    "k": top_k,  # Number of nearest neighbors to return
                }
            ],
            "select": "id,content",  # Fields to return (customize based on schema)
            "top": top_k,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Read the wrapped error after 'Failed to generate embedding for query:' — fix that underlying cause (401 => key, 404 => deployment name/base, 400 => api_version)
2. Test the embedding in isolation: litellm.embedding(model=emb_model, input=['ping'], **emb_config)
3. Verify the deployment name in Azure AI Foundry matches the model string after the '/' exactly
4. For throttling, retry with backoff or raise TPM limits on the deployment

Example fix

# before
litellm_embedding_config={'api_base': 'https://wrong-resource.cognitiveservices.azure.com/', 'api_key': key}

# after
litellm_embedding_config={'api_base': 'https://correct-resource.cognitiveservices.azure.com/', 'api_key': key, 'api_version': '2025-09-01'}

try:
    litellm.embedding(model=emb_model, input=['ping'], **emb_config)
except Exception as e:
    raise RuntimeError(f'Embedding precheck failed: {e}')

try:
    results = vector_store.search(vector_store_id=idx, query=q)
except Exception as e:
    if 'Failed to generate embedding for query' in str(e):
        # inspect inner cause; retry on throttling, fix config on auth/404
        if '429' in str(e) or 'throttl' in str(e).lower():
            time.sleep(backoff); results = vector_store.search(vector_store_id=idx, query=q)
        else:
            raise

When it happens

Trigger: Wrong embedding api_base/api_key in litellm_embedding_config; embedding deployment name not matching the model string; api_version rejected by the endpoint; per-call rate limits or throttling during search. The original exception text after the colon identifies which.

Common situations: Embedding deployment in a different Azure resource than configured; key rotated in the portal but not in config; using an OpenAI model string with Azure-style config or vice versa; hitting token limits with a very long query.

Related errors

• embedding_model is required in litellm_params for Azure AI S
• embedding_config is required in litellm_params for Azure AI
• Failed to parse raw Azure embedding response: {json_error}
• embedding_response is not an instance of EmbeddingResponse
• api_base is None. Please set AZURE_AI_API_BASE or dynamicall

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/add3af498f904fbc. Report an issue: GitHub.