Tencent/WeKnora · error

failed to do bulk: %w

Error message

failed to do bulk: %w

What it means

BatchSave wraps any error returned by the go-elasticsearch v8 bulk IndexRequest.Do call with 'failed to do bulk: %w'. The bulk API sends all indexed documents to Elasticsearch in one request, so a failure here means the whole batch was rejected or the request failed (network, mapping, server error). The original ES error is preserved for unwrapping via errors.Is/errors.As.

Source

Thrown at internal/application/repository/retriever/elasticsearch/v8/repository.go:212

	log.Infof("[Elasticsearch] Batch saving %d indices", len(embeddingList))
	indexRequest := e.client.Bulk().Index(e.index)

	// Add each document to the bulk request
	for _, embedding := range embeddingList {
		embeddingDB := elasticsearchRetriever.ToDBVectorEmbedding(embedding, additionalParams)
		err := indexRequest.CreateOp(types.CreateOperation{Index_: &e.index}, embeddingDB)
		if err != nil {
			log.Errorf("[Elasticsearch] Failed to create bulk operation: %v", err)
			return fmt.Errorf("failed to create op: %w", err)
		}
		log.Debugf("[Elasticsearch] Added chunk ID %s to bulk request", embedding.ChunkID)
	}

	// Execute the bulk request
	_, err := indexRequest.Do(ctx)
	if err != nil {
		log.Errorf("[Elasticsearch] Failed to execute bulk operation: %v", err)
		return fmt.Errorf("failed to do bulk: %w", err)
	}

	log.Infof("[Elasticsearch] Successfully batch saved %d indices", len(embeddingList))
	return nil
}

// DeleteByChunkIDList removes documents from the index based on chunk IDs
// Returns an error if the delete operation fails
func (e *elasticsearchRepository) DeleteByChunkIDList(ctx context.Context, chunkIDList []string, dimension int, knowledgeType string) error {
	log := logger.GetLogger(ctx)
	if len(chunkIDList) == 0 {
		log.Warn("[Elasticsearch] Empty chunk ID list provided for deletion, skipping")
		return nil
	}

	log.Infof("[Elasticsearch] Deleting indices by chunk IDs, count: %d", len(chunkIDList))
	// Use DeleteByQuery to delete all documents matching the chunk IDs
	_, err := e.client.DeleteByQuery(e.index).Query(&types.Query{

View on GitHub (pinned to 988cbb0330)

Solutions

1. Check Elasticsearch health (GET /_cluster/health) and connectivity from the app host; verify host/credentials config used to build the client.
2. Inspect the wrapped error (errors.Unwrap or %v in logs) — a bulk response failure typically names the offending document and reason.
3. Verify the embedding dimension matches the index mapping (e.g. dense_vector dims); delete/recreate the index if the mapping changed.
4. Check disk watermark and index write block settings (GET /<index>/_settings) and clear blocks if present.
5. Retry the batch once connectivity is restored; the operation is idempotent per document ID.

Example fix

// before
err := retriever.BatchSave(ctx, items) // panic/log-and-ignore
// after
if err := retriever.BatchSave(ctx, items); err != nil {
	var esErr *elastic.Error
	if errors.As(err, &esErr) {
		log.Printf("ES bulk failed status=%d: %s", esErr.Status, esErr.Error())
	}
	return fmt.Errorf("batch save embeddings: %w", err)
}

if len(embeddingList) == 0 { return nil }
if err := pingElasticsearch(ctx, client); err != nil { return fmt.Errorf("es unavailable: %w", err) }

if err := retriever.BatchSave(ctx, items); err != nil {
	log.Errorf("es bulk save failed: %v", err) // wrapped root cause preserved
	return err
}

When it happens

Trigger: Calling BatchSave when the Elasticsearch cluster is unreachable, the index mapping rejects a document (e.g. embedding field type conflict), a document is malformed, or the bulk request Do(ctx) returns a connection/timeout/4xx-5xx error.

Common situations: Elasticsearch down or misconfigured host/port in env; version mismatch between go-elasticsearch v8 client and an older/newer server; embedding vector dimension mismatch with the index mapping causing mapper_parsing_exception; disk watermark exceeded leading to 429/403 rejection of writes.

Related errors

• failed to delete by query: %w
• rerank call failed: %w
• invalid search result format
• invalid hit object format
• invalid retriever type: %v

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/372feecb274f21c1. Report an issue: GitHub.