crewAIInc/crewAI · error · RuntimeError

Failed to generate embedding: {e}

Error message

Failed to generate embedding: {e}

What it means

EmbeddingService.embed_text catches every exception from calling the underlying embedding function on a single text and re-raises as RuntimeError('Failed to generate embedding: ...'). Common roots: provider API auth errors, rate limits, network failures, or a malformed text payload. Empty/whitespace-only text is short-circuited to [] before this path and only logs a warning.

Source

Thrown at lib/crewai-tools/src/crewai_tools/rag/embedding_service.py:270

        Returns:
            List of floats representing the embedding

        Raises:
            RuntimeError: If embedding generation fails
        """
        if not text or not text.strip():
            logger.warning("Empty text provided for embedding")
            return []

        try:
            # Use ChromaDB's embedding function interface
            embeddings = self._embedding_function([text])  # type: ignore
            return list(embeddings[0]) if embeddings else []

        except Exception as e:
            logger.error(f"Error generating embedding for text: {e}")
            raise RuntimeError(f"Failed to generate embedding: {e}") from e

    def embed_batch(self, texts: list[str]) -> list[list[float]]:
        """
        Generate embeddings for multiple texts.

        Args:
            texts: List of texts to embed

        Returns:
            List of embedding vectors

        Raises:
            RuntimeError: If embedding generation fails
        """
        if not texts:
            return []

        valid_texts = [text for text in texts if text and text.strip()]

View on GitHub (pinned to 754d7323be)

Solutions

1. Inspect e.__cause__: 401/403 -> fix API key; 429 -> back off and retry; timeout -> check network/proxy
2. Add retry with exponential backoff for 429/5xx provider errors
3. Ensure the provider key env var is set and current before ingestion runs
4. Batch via embed_batch instead of per-text calls to reduce request count and rate-limit pressure

Example fix

# before
vec = service.embed_text(text)
# after
for attempt in range(5):
    try:
        vec = service.embed_text(text)
        break
    except RuntimeError as e:
        if '429' in str(e.__cause__ or '') and attempt < 4:
            time.sleep(2 ** attempt); continue
        raise

if not text or not text.strip():
    return []  # skip empty, mirroring the service's own guard

def is_embeddable_text(t: str) -> bool:
    return bool(t and t.strip())

for attempt in range(4):
    try:
        vec = service.embed_text(text); break
    except RuntimeError as e:
        msg = str(e.__cause__ or '')
        if ('429' in msg or 'timeout' in msg.lower()) and attempt < 3:
            time.sleep(2 ** attempt); continue
        raise

When it happens

Trigger: Expired/invalid API key on the provider call; rate limiting (429) from OpenAI/other providers; transient network outage; text containing content the provider rejects; batch interface returning an unexpected shape.

Common situations: Long-running RAG ingestion exhausting quotas; rotated API keys not updated in env; flaky connectivity in containers; embedding at document count spikes.

Related errors

• Failed to generate batch embeddings: {e}
• CrewAI embedding providers not available. Make sure crewai i
• Failed to initialize {self.config.provider} embedding servic
• Invalid configuration for embedding provider '{provider}':\n
• Project name '{name}' would generate invalid Python class na

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/d04f65e388afb17f. Report an issue: GitHub.