BerriAI/litellm · error · ValueError

Failed to generate embedding: {e}

Error message

Failed to generate embedding: {e}

What it means

Before a semantic-cache lookup or store, the cache embeds the prompt by calling litellm.aembedding with the configured embedding_model (default text-embedding-ada-002). Any failure in that embedding call — auth error, unknown model, provider outage, malformed response — is caught, logged via print_verbose, and re-raised as ValueError('Failed to generate embedding: ...') with the original cause chained. The original exception text is embedded in the message and is the key to diagnosis.

Source

Thrown at litellm/caching/redis_semantic_cache.py:510

        router: Final = resolve_embedding_router(self.embedding_model, llm_router, llm_model_list)
        try:
            if router is not None:
                embedding_response = await router.aembedding(
                    model=self.embedding_model,
                    input=prompt,
                    cache={"no-store": True, "no-cache": True},
                    metadata=build_router_embedding_metadata(metadata),
                )
            else:
                embedding_response = await litellm.aembedding(
                    model=self.embedding_model,
                    input=prompt,
                    cache={"no-store": True, "no-cache": True},
                )
            return embedding_response["data"][0]["embedding"]
        except Exception as e:
            print_verbose(f"Error generating async embedding: {e}")
            raise ValueError(f"Failed to generate embedding: {e}") from e

    async def async_set_cache(self, key: str, value: object, **kwargs) -> None:
        """
        Asynchronously store a value in the semantic cache.

        Args:
            key: The cache key used to isolate semantic cache entries
            value: The response value to cache
            **kwargs: Additional arguments including 'messages' for the prompt
                and optional 'ttl' for time-to-live
        """
        print_verbose(f"Async Redis semantic-cache set_cache, kwargs: {kwargs}")

        try:
            prompt: Final = self._get_prompt_from_kwargs(**kwargs)
            if prompt is None:
                print_verbose("No prompt provided for semantic caching")
                return

View on GitHub (pinned to 6c2dcb801b)

Solutions

  1. Read the chained cause in the message — it names the real failure (401, model not found, timeout)
  2. Set a valid key for the embedding model, e.g. os.environ['OPENAI_API_KEY'] = 'sk-...' when using the default ada-002
  3. Point embedding_model at a model you actually serve, e.g. embedding_model='bedrock/...'|'azure/...'
  4. Verify litellm.aembedding(model=..., input=['ping']) works standalone before enabling semantic caching

Example fix

# before
cache = RedisSemanticCache(redis_url=url, similarity_threshold=0.8)  # no OPENAI_API_KEY

# after
os.environ['OPENAI_API_KEY'] = 'sk-...'
cache = RedisSemanticCache(redis_url=url, similarity_threshold=0.8,
                           embedding_model='text-embedding-ada-002')
Defensive patterns

Strategy: try-catch

Validate before calling

import litellm

async def embedding_ok(model: str) -> bool:
    try:
        r = await litellm.aembedding(model=model, input=['ping'], cache={'no-store': True})
        return bool(r['data'][0]['embedding'])
    except Exception:
        return False

# gate semantic caching on this check at startup

Try / catch

try:
    resp = await litellm.acompletion(...)
except ValueError as e:
    if 'Failed to generate embedding' in str(e):
        logger.warning('Semantic cache embedding failed; retrying with cache disabled')
        kwargs['cache']['no-cache'] = True
        resp = await litellm.acompletion(...)
    else:
        raise

Prevention

When it happens

Trigger: Semantic caching enabled without a valid OPENAI_API_KEY (for the default ada-002 model); embedding_model set to a model the configured providers don't serve; embedding provider rate-limited or down; router missing a deployment for the embedding model when aembedding is dispatched through it.

Common situations: Users enable redis-semantic caching for chat completions but never configure embedding credentials; switching embedding models without updating vector_size/index dims; OpenAI key expired or scoped without embedding access.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/da59431fe59a15a5. Report an issue: GitHub.