{"record":{"id":"724fd0c21732df8b","repo":"NationalSecurityAgency/ghidra","slug":"vector-document-does-not-exist-for-id","errorCode":null,"errorMessage":"vector document does not exist for id=","messagePattern":"vector document does not exist for id=","errorType":"exception","errorClass":"ElasticException","httpStatus":null,"severity":"critical","filePath":"Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java","lineNumber":707,"sourceCode":"\t\tbuffer.append(\" ] }\");\n\t\tJsonObject resp =\n\t\t\tconnection.executeStatement(ElasticConnection.GET, \"vector/_mget\", buffer.toString());\n\t\tJsonArray docs = (JsonArray) resp.get(\"docs\");\n\t\tchar[] vectorDecodeBuffer = Base64VectorFactory.allocateBuffer();\n\t\tfor (int i = 0; i < maxDocuments; ++i) {\n\t\t\tif (!iter2.hasNext()) {\n\t\t\t\tbreak;\n\t\t\t}\n\t\t\tvecRes = iter2.next();\n\t\t\tJsonObject oneResp = (JsonObject) docs.get(i);\n\t\t\tString matchId = oneResp.get(\"_id\").getAsString();\n\t\t\tlong matchIdVal = Base64Lite.decodeLongBase64(matchId);\n\t\t\tif (matchIdVal != vecRes.vectorid) {\n\t\t\t\tthrow new ElasticException(\"Mismatch in vectorid\");\n\t\t\t}\n\t\t\tJsonElement source = oneResp.get(\"_source\");\n\t\t\tif (ElasticConnection.isNull(source)) {\n\t\t\t\tthrow new ElasticException(\"vector document does not exist for id=\" + matchId);\n\t\t\t}\n\t\t\tStringReader reader =\n\t\t\t\tnew StringReader(((JsonObject) source).get(\"features\").getAsString());\n\t\t\ttry {\n\t\t\t\tvecRes.vec = vectorFactory.restoreVectorFromBase64(reader, vectorDecodeBuffer);\n\t\t\t}\n\t\t\tcatch (IOException e) {\n\t\t\t\tthrow new ElasticException(e.getMessage());\n\t\t\t}\n\t\t}\n\t}\n\n\t/**\n\t * Given a list of FunctionDescriptions, fill in the matching SignatureRecords\n\t * @param listFunctions is the list of functions\n\t * @param manager is the FunctionDescription container\n\t * @throws ElasticException for communication problems with the server\n\t */","sourceCodeStart":689,"sourceCodeEnd":725,"githubUrl":"https://github.com/NationalSecurityAgency/ghidra/blob/d5f144c24d6bc53c9cbf4448c6d11143e7696206/Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java#L689-L725","documentation":"In fetchVectors, after the vector document's _id matches, its _source is checked for null. A null/JsonNull _source means ES returned a doc envelope for the id but the vector document body is absent: the vector was deleted or never indexed while a result still references it. Index-consistency break; the features field cannot be read.","triggerScenarios":"A VectorResult references a vectorid, vector/_mget returns a doc with that _id but _source is null. Happens after partial delete of vector docs, interrupted ingest, replication lag, or corruption.","commonSituations":"Interrupted ingest that recorded meta but not all vectors; a delete leaving dangling references; replica inconsistency during node failure.","solutions":["Retry after a short delay to allow refresh/replication to converge.","If persistent, locate the missing vector document and re-ingest it, or rebuild the vector index.","Verify cluster health (yellow/red, unassigned shards) which typically accompanies such gaps."],"exampleFix":"// before\ndb.fetchVectors(it1, it2, n);\n// after\ntry { db.fetchVectors(it1, it2, n); }\ncatch (ElasticException e) {\n  if (e.getMessage().startsWith(\"vector document does not exist\")) { sleep(2000); db.fetchVectors(it1copy, it2copy, n); return; }\n  throw e;\n}","handlingStrategy":"retry","validationCode":"public static boolean vectorPresent(ElasticConnection c, String b64Id) throws ElasticException {\n    JsonObject r = c.executeStatementExpectFailure(ElasticConnection.GET, \"vector/_doc/\" + b64Id, \"\");\n    JsonElement found = ((JsonObject) r).get(\"found\");\n    return found != null && found.getAsString().equals(\"true\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    db.fetchVectors(it1, it2, n);\n} catch (ElasticException e) {\n    if (!e.getMessage().startsWith(\"vector document does not exist\")) throw e;\n    Thread.sleep(2000);\n    db.fetchVectors(copy(it1), copy(it2), n);\n}","preventionTips":["Allow refresh/replication to converge before reading freshly written vectors.","Watch cluster health for unassigned shards that cause missing-source responses.","Re-ingest missing vectors or rebuild the index if gaps persist."],"tags":["elasticsearch","data-integrity","bsim","consistency"],"backgroundTag":null,"analyzedSha":"d5f144c24d6bc53c9cbf4448c6d11143e7696206","analyzedAt":"2026-08-14T01:00:57.564Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}