crewAIInc/crewAI · error · RuntimeError

Failed to generate batch embeddings: {e}

Error message

Failed to generate batch embeddings: {e}

What it means

EmbeddingService.embed_batch validates texts, then slices them into groups of config.batch_size and calls the provider embedding function per batch; any exception (auth, rate limit, network, oversized batch rejected by provider, unexpected return shape) is caught and re-raised as RuntimeError('Failed to generate batch embeddings: ...'). The cause chain preserves the provider error.

Source

Thrown at lib/crewai-tools/src/crewai_tools/rag/embedding_service.py:306

        valid_texts = [text for text in texts if text and text.strip()]
        if not valid_texts:
            logger.warning("No valid texts provided for batch embedding")
            return []

        try:
            # Process in batches to avoid API limits
            all_embeddings: list[list[float]] = []

            for i in range(0, len(valid_texts), self.config.batch_size):
                batch = valid_texts[i : i + self.config.batch_size]
                batch_embeddings = self._embedding_function(batch)  # type: ignore
                all_embeddings.extend(list(e) for e in batch_embeddings)

            return all_embeddings

        except Exception as e:
            logger.error(f"Error generating batch embeddings: {e}")
            raise RuntimeError(f"Failed to generate batch embeddings: {e}") from e

    def get_embedding_dimension(self) -> int | None:
        """
        Get the dimension of embeddings produced by this service.

        Returns:
            Embedding dimension or None if unknown
        """
        # Try to get dimension by generating a test embedding
        try:
            test_embedding = self.embed_text("test")
            return len(test_embedding) if test_embedding else None
        except Exception:
            logger.warning("Could not determine embedding dimension")
            return None

    def validate_connection(self) -> bool:
        """

View on GitHub (pinned to 754d7323be)

Solutions

1. Lower config.batch_size to a provider-safe value (e.g. 64-256)
2. Read e.__cause__ to distinguish auth (fix key) vs 429 (back off) vs validation (inspect texts)
3. Retry the failing batch alone to identify a poison text; sanitize/remove it
4. Stagger concurrent ingestion workers or add a global rate limiter

Example fix

# before
vecs = service.embed_batch(all_texts)
# after
vecs = []
for i in range(0, len(all_texts), 64):
    chunk = all_texts[i:i+64]
    try:
        vecs.extend(service.embed_batch(chunk))
    except RuntimeError:
        time.sleep(2); vecs.extend(service.embed_batch(chunk))  # naive retry for 429

BATCH_SIZE = 64  # provider-safe value
texts = [t for t in texts if t and t.strip()]  # drop empties that can poison batches
for i in range(0, len(texts), BATCH_SIZE):
    ...

def is_valid_batch(texts: list[str], limit: int) -> bool:
    return all(t and t.strip() for t in texts) and len(texts) <= limit

results = []
for i in range(0, len(texts), 64):
    for attempt in range(3):
        try:
            results.extend(service.embed_batch(texts[i:i+64])); break
        except RuntimeError:
            if attempt == 2: raise
            time.sleep(2 ** attempt)

When it happens

Trigger: batch_size larger than the provider's per-request limit (e.g. >2048 inputs for OpenAI); a 429 triggered by rapid sequential batches; one malformed text in the batch failing the whole call; provider returning fewer embeddings than inputs.

Common situations: Bulk document ingestion in RAG pipelines; batch_size copied from docs of a different provider; running ingestion concurrently from multiple workers tripping shared rate limits.

Related errors

• Failed to generate embedding: {e}
• CrewAI embedding providers not available. Make sure crewai i
• Failed to initialize {self.config.provider} embedding servic
• Invalid configuration for embedding provider '{provider}':\n
• Client is not initialized

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/ad550b851ae75096. Report an issue: GitHub.