Tencent/WeKnora · error

failed to index chunk: %w

Error message

failed to index chunk: %w

What it means

CreateFAQEntry indexes the chunk within a hard 5-second budget (faqCreateIndexBudget) by embedding it via indexFAQChunks; on failure it deletes the stored chunk (best-effort rollback) and returns this error. It means embedding/indexing failed or exceeded the 5s timeout, most often because the embedding backend is slow, degraded, or unreachable. Note the rollback may fail, leaving a stored-state chunk that still triggers duplicate checks.

Source

Thrown at internal/application/service/knowledge_faq.go:243

	}

	// 保存chunk
	if err := s.chunkService.CreateChunks(ctx, []*types.Chunk{chunk}); err != nil {
		return nil, fmt.Errorf("failed to create chunk: %w", err)
	}

	// 索引chunk：交互式创建给索引步骤设硬上限，避免 embedding 抖动把请求拖长
	indexCtx, cancelIndex := context.WithTimeout(ctx, faqCreateIndexBudget)
	indexErr := s.indexFAQChunks(indexCtx, kb, faqKnowledge, []*types.Chunk{chunk}, embeddingModel, true, false)
	cancelIndex()
	if indexErr != nil {
		// 如果索引失败，删除已创建的chunk。回滚失败会留下一条 stored 状态的
		// 残留：它不出现在列表里，却会被重复校验命中，因此必须告警而非静默。
		if delErr := s.chunkService.DeleteChunk(ctx, chunk.ID); delErr != nil {
			logger.Errorf(ctx,
				"CreateFAQEntry: rollback failed, chunk %s left in stored state: %v", chunk.ID, delErr)
		}
		return nil, fmt.Errorf("failed to index chunk: %w", indexErr)
	}

	// 更新chunk状态为已索引
	chunk.Status = int(types.ChunkStatusIndexed)
	if err := s.chunkService.UpdateChunk(ctx, chunk); err != nil {
		return nil, fmt.Errorf("failed to update chunk status: %w", err)
	}

	// Build tag seq_id map for conversion
	tagSeqIDMap := make(map[string]int64)
	if chunk.TagID != "" {
		tag, tagErr := s.tagRepo.GetByID(ctx, tenantID, chunk.TagID)
		if tagErr == nil && tag != nil {
			tagSeqIDMap[tag.ID] = tag.SeqID
		}
	}

	// 转换为FAQEntry返回

View on GitHub (pinned to 988cbb0330)

Solutions

1. Check embedding service health/latency and API credentials; retry after the provider recovers.
2. Verify vector-store (e.g. Elasticsearch/Milvus) connectivity if the embedding call itself succeeded.
3. Look for rollback-failure warnings ('rollback failed, chunk ... left in stored state') and manually delete stored-state residue before retrying, or duplicates will be blocked.
4. For bulk work, use import/bulk APIs which keep the full retry budget instead of the 5s interactive cap.
5. Reduce FAQ answer size or switch to a faster/nearest embedding model.

Example fix

// before: embedding provider misconfigured
// EMBEDDING_API_KEY="" (empty)
// after: set valid credentials and verify before creating entries
export EMBEDDING_API_KEY=sk-...
curl -s $EMBEDDING_ENDPOINT/health  # must return 200 before retry

// verify embedding backend before creating entries
resp, err := http.Get(os.Getenv("EMBEDDING_ENDPOINT") + "/health")
if err != nil || resp.StatusCode != 200 {
	return errors.New("embedding service unhealthy; postpone FAQ creation")
}

entry, err := svc.CreateFAQEntry(ctx, kbID, payload)
if err != nil && strings.Contains(err.Error(), "failed to index chunk") {
	if errors.Is(err, context.DeadlineExceeded) {
		// embedding exceeded the 5s budget; retry after backend recovers
		time.Sleep(5 * time.Second)
		entry, err = svc.CreateFAQEntry(ctx, kbID, payload)
	}
	// also check logs for 'rollback failed' and clean stored-state residue
}

When it happens

Trigger: CreateFAQEntry where embedding takes >5s (embedding service latency/spike), the embedding API returns an error (quota, auth, model unavailable), the vector store write fails, or the parent context is cancelled.

Common situations: Overloaded or rate-limited embedding provider; wrong embedding API key/endpoint in environment config; network partition between WeKnora and the embedding/vector services; large FAQ answers inflating embedding time; upstream client retries piling concurrent creates.

Related errors

• failed to get embedding model: %w
• E2B timeout must be at least one second
• E2B backend does not support NeverTimeout
• timeout cannot be negative
• failed to ensure FAQ knowledge: %w

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/7e9a70ec566fde9c. Report an issue: GitHub.