{"record":{"id":"e9df2612482da68b","repo":"NationalSecurityAgency/ghidra","slug":"mismatch-in-decrementvectorcounters","errorCode":null,"errorMessage":"Mismatch in decrementVectorCounters","messagePattern":"Mismatch in decrementVectorCounters","errorType":"exception","errorClass":"ElasticException","httpStatus":null,"severity":"error","filePath":"Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java","lineNumber":1828,"sourceCode":"\t\t\tbuffer.append(\"\\\"_id\\\": \\\"\");\n\t\t\tBase64Lite.encodeLongBase64(buffer, entry.id);\n\t\t\tbuffer.append(\"\\\", \\\"retry_on_conflict\\\": 5 } }\\n\");\n\t\t\tbuffer.append(\n\t\t\t\t\"{ \\\"script\\\": { \\\"inline\\\": \\\"if ((ctx._source.count -= params.count) <=0) { ctx.op = \\\\\\\"delete\\\\\\\" }\\\", \");\n\t\t\tbuffer.append(\"\\\"params\\\": { \\\"count\\\": \").append(entry.count).append(\"} } }\\n\");\n\t\t}\n\t\tJsonObject resp = connection.executeBulk(\"/_bulk\", buffer.toString());\n\t\tJsonArray items = (JsonArray) resp.get(\"items\");\n\t\tfor (int i = 0; i < maxVectors; ++i) {\n\t\t\tif (!iter2.hasNext()) {\n\t\t\t\tbreak;\n\t\t\t}\n\t\t\tIdHistogram entry = iter2.next();\n\t\t\tJsonObject item = (JsonObject) items.get(i);\n\t\t\tJsonObject update = (JsonObject) item.get(\"update\");\n\t\t\tlong id = Base64Lite.decodeLongBase64(update.get(\"_id\").getAsString());\n\t\t\tif (id != entry.id) {\n\t\t\t\tthrow new ElasticException(\"Mismatch in decrementVectorCounters\");\n\t\t\t}\n\t\t\tif (\"deleted\".equals(ElasticConnection.convertToString(update.get(\"result\")))) {\n\t\t\t\tdeleteList.add(entry);\t\t\t\t// Mark this vector for full deletion\n\t\t\t}\n\t\t}\n\t}\n\n\t/**\n\t * Delete vector documents in bulk. This assumes multiplicity counts in the \"meta\" documents\n\t * have already been checked, and these vectors are scheduled for full document deletion.\n\t * Vectors are presented as an iterator to IdHistograms. One bulk deletion request is\n\t * submitted containing vectors up to a given maximum number. The iterator is advanced by\n\t * the number submitted\n\t * @param iter is an iterator over records containing the id's to delete\n\t * @param maxVectors is the maximum number to delete for this window\n\t * @throws ElasticException for communication problems with the server\n\t */\n\tprivate void deleteRawVectors(Iterator<IdHistogram> iter, int maxVectors)","sourceCodeStart":1810,"sourceCodeEnd":1846,"githubUrl":"https://github.com/NationalSecurityAgency/ghidra/blob/d5f144c24d6bc53c9cbf4448c6d11143e7696206/Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java#L1810-L1846","documentation":"Thrown in decrementVectorCounters when the _id decoded from a bulk update response item does not match the expected vector id from the iterator. Elasticsearch's bulk API guarantees responses are returned in the same order as requests; a mismatch indicates a protocol contract violation, response body corruption, or a desynchronization between the two parallel iterators (iter1 for building requests, iter2 for matching responses).","triggerScenarios":"During executable deletion, decrementVectorCounters sends a bulk update to repository_meta with retry_on_conflict=5, then iterates the response items alongside iter2. Base64Lite.decodeLongBase64(update.get(\"_id\").getAsString()) for item i is compared against entry.id from iter2; any difference triggers the exception.","commonSituations":"Should never occur under normal operation. If seen, it points to: a proxy or load balancer reordering or corrupting bulk response items; a bug in how iter1 and iter2 are managed by the caller; an Elasticsearch bulk API contract violation in a non-standard server fork.","solutions":["Report as a bug — this guard should never fire with a compliant Elasticsearch server; investigate proxy or load-balancer interference with bulk NDJSON responses.","Verify that the two iterators (iter1, iter2) passed to decrementVectorCounters point to the same underlying list and are at the same position.","Check for any HTTP intermediary (reverse proxy, API gateway) that might buffer or reorder bulk responses.","Confirm the Elasticsearch server version adheres to the standard bulk response ordering contract."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    database.query(query);\n} catch (ElasticException e) {\n    if (e.getMessage().contains(\"Mismatch in decrementVectorCounters\")) {\n        // Should never occur — indicates protocol violation or proxy interference\n        Msg.error(this, \"Bulk response order mismatch — check proxy/load-balancer: \" + e.getMessage());\n    }\n    throw e;\n}","preventionTips":["Ensure no HTTP proxy or load balancer reorders or buffers bulk NDJSON responses.","Verify the two iterators passed to decrementVectorCounters are backed by the same list.","Use a direct connection to Elasticsearch for bulk operations, bypassing any intermediary."],"tags":["elasticsearch","bsim","bulk","delete","defensive-guard","protocol-violation","meta"],"backgroundTag":null,"analyzedSha":"d5f144c24d6bc53c9cbf4448c6d11143e7696206","analyzedAt":"2026-08-14T01:00:57.564Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}