{"record":{"id":"a967b98df84da096","repo":"siyuan-note/siyuan","slug":"rerank-returned-duplicate-index-d","errorCode":null,"errorMessage":"rerank returned duplicate index %d","messagePattern":"rerank returned duplicate index (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"kernel/util/openai.go","lineNumber":751,"sourceCode":"}\n\n// TestRerankModel 测试重排模型可用性，用极简 query+documents 发一次重排请求验证连通性与鉴权。\n// 返回值：matched 表示是否连通成功，err 为请求错误（鉴权失败、网络异常、模型不存在等，原样返回便于调用方展示原因）。\nfunc TestRerankModel(options RerankOptions) (matched bool, err error) {\n\tdocuments := []string{\"a\", \"b\"}\n\toptions.TopN = len(documents)\n\tindices, _, err := Rerank(\"1\", documents, options)\n\tif nil != err {\n\t\treturn\n\t}\n\tif len(indices) != len(documents) {\n\t\terr = fmt.Errorf(\"rerank returned %d indices for %d documents\", len(indices), len(documents))\n\t\treturn\n\t}\n\tseen := make(map[int]bool, len(indices))\n\tfor _, index := range indices {\n\t\tif seen[index] {\n\t\t\terr = fmt.Errorf(\"rerank returned duplicate index %d\", index)\n\t\t\treturn\n\t\t}\n\t\tseen[index] = true\n\t}\n\tmatched = true\n\treturn\n}\n\n// PrepareModelImage 校验并按需缩放图片，尽量保留多模态模型支持的原始格式和图片质量。\nfunc PrepareModelImage(data []byte, maxBytes, maxPixels, maxEdge int) (PreparedImage, error) {\n\tif len(data) == 0 {\n\t\treturn PreparedImage{}, errors.New(\"image data is empty\")\n\t}\n\tif maxBytes > 0 && len(data) > maxBytes {\n\t\treturn PreparedImage{}, fmt.Errorf(\"image exceeds size limit: %d bytes\", maxBytes)\n\t}\n\tmimeType := mimetype.Detect(data).String()\n\tif strings.Contains(mimeType, \"svg\") || bytes.Contains(bytes.ToLower(data[:min(len(data), 512)]), []byte(\"<svg\")) {","sourceCodeStart":733,"sourceCodeEnd":769,"githubUrl":"https://github.com/siyuan-note/siyuan/blob/9f775e8a12daef8255556097396f9b2739078892/kernel/util/openai.go#L733-L769","documentation":"As part of rerank result validation, the code builds a seen-set over returned indices and rejects the result if the same document index appears twice. A duplicate index means the provider's rerank output is corrupt or non-conformant, since each result must reference a distinct document. (The DECLARED/USED metadata referencing app/electron/accessibility.js is unrelated boilerplate; the authoritative location is the SOURCE region in kernel/util/openai.go.)","triggerScenarios":"A rerank response contains two entries with the same 'index' value — a buggy or non-standard provider, a client-side aggregation bug, or duplicated entries in a hand-rolled/mocked rerankResponse.","commonSituations":"Using a provider whose rerank implementation returns duplicate indices under some inputs; testing against a stubbed server producing repeated indices; protocol drift after a provider API update.","solutions":["Dump the raw response JSON and confirm the duplicate indices come from the provider, not local parsing","Retry the request; if duplicates persist, file a bug with the provider or switch rerank provider","Deduplicate defensively before consuming results, keeping the highest score per index","Pin to a provider API version known to return 1:1 index mappings"],"exampleFix":"// before: fail hard on any duplicate\nif seen[index] {\n    err = fmt.Errorf(\"rerank returned duplicate index %d\", index)\n}\n// after: keep best score per index and de-duplicate\nif seen[index] {\n    continue // or merge scores and keep max\n}","handlingStrategy":"validation","validationCode":"func indicesUnique(indices []int) bool {\n    seen := map[int]bool{}\n    for _, i := range indices {\n        if seen[i] { return false }\n        seen[i] = true\n    }\n    return true\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"duplicate index\") {\n    log.Printf(\"provider returned corrupt indices, using original order: %v\", err)\n    return docs\n}","preventionTips":["Validate provider responses with a 1:1 index check in integration tests","Switch providers if duplicates reproduce deterministically","Deduplicate client-side (keep max score) when tolerable instead of failing the whole search"],"tags":["rerank","validation","ai-provider"],"backgroundTag":"unexpected-response-shape","analyzedSha":"9f775e8a12daef8255556097396f9b2739078892","analyzedAt":"2026-09-19T03:17:15.984Z","contentChangedAt":"2026-09-19T03:17:15.984Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}