{"record":{"id":"e4e8202c17e5d528","repo":"NationalSecurityAgency/ghidra","slug":"mismatch-in-vectorid","errorCode":null,"errorMessage":"Mismatch in vectorid","messagePattern":"Mismatch in vectorid","errorType":"exception","errorClass":"ElasticException","httpStatus":null,"severity":"critical","filePath":"Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java","lineNumber":703,"sourceCode":"\t\t\tbuffer.append(\", \\\"\");\n\t\t\tBase64Lite.encodeLongBase64(buffer, vecRes.vectorid);\n\t\t\tbuffer.append('\\\"');\n\t\t}\n\t\tbuffer.append(\" ] }\");\n\t\tJsonObject resp =\n\t\t\tconnection.executeStatement(ElasticConnection.GET, \"vector/_mget\", buffer.toString());\n\t\tJsonArray docs = (JsonArray) resp.get(\"docs\");\n\t\tchar[] vectorDecodeBuffer = Base64VectorFactory.allocateBuffer();\n\t\tfor (int i = 0; i < maxDocuments; ++i) {\n\t\t\tif (!iter2.hasNext()) {\n\t\t\t\tbreak;\n\t\t\t}\n\t\t\tvecRes = iter2.next();\n\t\t\tJsonObject oneResp = (JsonObject) docs.get(i);\n\t\t\tString matchId = oneResp.get(\"_id\").getAsString();\n\t\t\tlong matchIdVal = Base64Lite.decodeLongBase64(matchId);\n\t\t\tif (matchIdVal != vecRes.vectorid) {\n\t\t\t\tthrow new ElasticException(\"Mismatch in vectorid\");\n\t\t\t}\n\t\t\tJsonElement source = oneResp.get(\"_source\");\n\t\t\tif (ElasticConnection.isNull(source)) {\n\t\t\t\tthrow new ElasticException(\"vector document does not exist for id=\" + matchId);\n\t\t\t}\n\t\t\tStringReader reader =\n\t\t\t\tnew StringReader(((JsonObject) source).get(\"features\").getAsString());\n\t\t\ttry {\n\t\t\t\tvecRes.vec = vectorFactory.restoreVectorFromBase64(reader, vectorDecodeBuffer);\n\t\t\t}\n\t\t\tcatch (IOException e) {\n\t\t\t\tthrow new ElasticException(e.getMessage());\n\t\t\t}\n\t\t}\n\t}\n\n\t/**\n\t * Given a list of FunctionDescriptions, fill in the matching SignatureRecords","sourceCodeStart":685,"sourceCodeEnd":721,"githubUrl":"https://github.com/NationalSecurityAgency/ghidra/blob/d5f144c24d6bc53c9cbf4448c6d11143e7696206/Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java#L685-L721","documentation":"Internal consistency check in fetchVectors: after a vector/_mget multi-get, each returned document's base64 _id is decoded and compared to VectorResult.vectorid at the same position. A mismatch means the returned docs are not in the requested order/id. ES _mget preserves order, so this points at a reordering proxy/gateway, a non-ES backend, a plugin, or index corruption.","triggerScenarios":"fetchVectors reading docs[i] whose decoded _id != the i-th vectorid. Caused by a proxy rewriting multi-get responses, an OpenSearch/plugin variant, or corruption returning wrong ids for the vector index.","commonSituations":"ES behind a gateway that reorders _mget; talking to a non-standard ES fork; rare corruption after replication fault.","solutions":["Reproduce with a direct curl vector/_mget bypassing the proxy to confirm ES preserves order.","Re-run in smaller batches, or match docs to VectorResults by id rather than position when post-processing.","If corruption is confirmed, reindex the BSim vector index from a known-good source."],"exampleFix":"// before: positional trust of _mget order (internal to fetchVectors)\n// after (defensive wrapper)\ntry { db.fetchVectors(it1, it2, n); }\ncatch (ElasticException e) { if (e.getMessage().equals(\"Mismatch in vectorid\")) { db.fetchVectors(it1copy, it2copy, n); return; } throw e; }","handlingStrategy":"fallback","validationCode":"// Internal ordering assertion; mitigate by verifying _mget order directly.\npublic static boolean vectorMgetOrdered(String url) throws Exception {\n    HttpURLConnection h = (HttpURLConnection) URI.create(url + \"/vector/_mget\").toURL().openConnection();\n    h.setRequestMethod(\"GET\"); h.setDoOutput(true);\n    try (Writer w = new OutputStreamWriter(h.getOutputStream())) { w.write(\"{\\\"ids\\\":[\\\"AAAB\\\",\\\"AAAC\\\"]}\"); }\n    JsonObject r = JsonParser.parseReader(new InputStreamReader(h.getInputStream())).getAsJsonObject();\n    return ((JsonArray) r.get(\"docs\")).get(0).getAsJsonObject().get(\"_id\").getAsString().equals(\"AAAB\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    db.fetchVectors(it1, it2, n);\n} catch (ElasticException e) {\n    if (!e.getMessage().equals(\"Mismatch in vectorid\")) throw e;\n    fetchVectorsSingly(it1, n); // order-independent single gets\n}","preventionTips":["Keep ES _mget responses free of reordering proxies.","Re-verify _mget ordering semantics after any ES/OpenSearch version or plugin change.","Treat recurring vectorid mismatches as corruption and reindex."],"tags":["elasticsearch","data-integrity","bsim","consistency"],"backgroundTag":null,"analyzedSha":"d5f144c24d6bc53c9cbf4448c6d11143e7696206","analyzedAt":"2026-08-14T01:00:57.564Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}