siyuan-note/siyuan · error

rerank returned duplicate index

Error message

rerank returned duplicate index %d

What it means

As part of rerank result validation, the code builds a seen-set over returned indices and rejects the result if the same document index appears twice. A duplicate index means the provider's rerank output is corrupt or non-conformant, since each result must reference a distinct document. (The DECLARED/USED metadata referencing app/electron/accessibility.js is unrelated boilerplate; the authoritative location is the SOURCE region in kernel/util/openai.go.)

Solutions

1. Dump the raw response JSON and confirm the duplicate indices come from the provider, not local parsing
2. Retry the request; if duplicates persist, file a bug with the provider or switch rerank provider
3. Deduplicate defensively before consuming results, keeping the highest score per index
4. Pin to a provider API version known to return 1:1 index mappings

Example fix

// before: fail hard on any duplicate
if seen[index] {
    err = fmt.Errorf("rerank returned duplicate index %d", index)
}
// after: keep best score per index and de-duplicate
if seen[index] {
    continue // or merge scores and keep max
}

func indicesUnique(indices []int) bool {
    seen := map[int]bool{}
    for _, i := range indices {
        if seen[i] { return false }
        seen[i] = true
    }
    return true
}

if err != nil && strings.Contains(err.Error(), "duplicate index") {
    log.Printf("provider returned corrupt indices, using original order: %v", err)
    return docs
}

When it happens

Trigger: A rerank response contains two entries with the same 'index' value — a buggy or non-standard provider, a client-side aggregation bug, or duplicated entries in a hand-rolled/mocked rerankResponse.

Common situations: Using a provider whose rerank implementation returns duplicate indices under some inputs; testing against a stubbed server producing repeated indices; protocol drift after a provider API update.

Related errors

• rerank returned indices for documents
• invalid AI provider HTTP headers
• rerank HTTP
• rerank response missing results
• 106

AI-assisted analysis of siyuan-note/siyuan@9f775e8a12 (2026-09-19). Data as JSON: /api/errors/a967b98df84da096. Report an issue: GitHub.

Appendix: source

Thrown at kernel/util/openai.go:751

}

// TestRerankModel 测试重排模型可用性，用极简 query+documents 发一次重排请求验证连通性与鉴权。
// 返回值：matched 表示是否连通成功，err 为请求错误（鉴权失败、网络异常、模型不存在等，原样返回便于调用方展示原因）。
func TestRerankModel(options RerankOptions) (matched bool, err error) {
	documents := []string{"a", "b"}
	options.TopN = len(documents)
	indices, _, err := Rerank("1", documents, options)
	if nil != err {
		return
	}
	if len(indices) != len(documents) {
		err = fmt.Errorf("rerank returned %d indices for %d documents", len(indices), len(documents))
		return
	}
	seen := make(map[int]bool, len(indices))
	for _, index := range indices {
		if seen[index] {
			err = fmt.Errorf("rerank returned duplicate index %d", index)
			return
		}
		seen[index] = true
	}
	matched = true
	return
}

// PrepareModelImage 校验并按需缩放图片，尽量保留多模态模型支持的原始格式和图片质量。
func PrepareModelImage(data []byte, maxBytes, maxPixels, maxEdge int) (PreparedImage, error) {
	if len(data) == 0 {
		return PreparedImage{}, errors.New("image data is empty")
	}
	if maxBytes > 0 && len(data) > maxBytes {
		return PreparedImage{}, fmt.Errorf("image exceeds size limit: %d bytes", maxBytes)
	}
	mimeType := mimetype.Detect(data).String()
	if strings.Contains(mimeType, "svg") || bytes.Contains(bytes.ToLower(data[:min(len(data), 512)]), []byte("<svg")) {

View on GitHub (pinned to 9f775e8a12)