NationalSecurityAgency/ghidra · critical · ElasticException

meta document does not exist for id=

Error message

meta document does not exist for id=

What it means

In fetchVectorCounts, after confirming the meta document's _id matches, its _source is checked for null. A null/JsonNull _source means ES returned a hit for the id but the document source is absent: the meta doc was deleted or never indexed while a vector referencing it still exists. This is an index-consistency break.

Source

Thrown at Ghidra/Features/BSim/src/main/java/ghidra/features/bsim/query/elastic/ElasticDatabase.java:648

		}
		buffer.append(" ] }");
		JsonObject resp =
			connection.executeStatement(ElasticConnection.GET, "meta/_mget", buffer.toString());
		JsonArray docs = (JsonArray) resp.get("docs");
		for (int i = 0; i < maxDocuments; ++i) {
			if (!iter2.hasNext()) {
				break;
			}
			vecRes = iter2.next();
			JsonObject oneResp = (JsonObject) docs.get(i);
			String matchId = oneResp.get("_id").getAsString();
			long matchIdVal = Base64Lite.decodeLongBase64(matchId);
			if (matchIdVal != vecRes.vectorid) {
				throw new ElasticException("Mismatch in metaid");
			}
			JsonElement source = oneResp.get("_source");
			if (ElasticConnection.isNull(source)) {
				throw new ElasticException("meta document does not exist for id=" + matchId);
			}
			long count = ((JsonObject) source).get("count").getAsLong();
			totalCount += count;
			vecRes.hitcount = (int) count;
		}
		return totalCount;
	}

	/**
	 * Fetch vectors in bulk from the database, given a list of VectorResults with the vector ids
	 * The vector documents are queried, then the resulting LSHVector objects are filled
	 * in for the VectorResults by parsing the documents. Two iterators pointing to the same list
	 * of VectorResults are required, one for building the query, one for filling in the LSHVectors.
	 * If no exception is thrown, both iterators are advanced the same number of times.
	 * @param iter1 is the iterator to VectorResults to fill in
	 * @param iter2 is a copy of the first iterator
	 * @param maxDocuments is the maximum number of documents to query for
	 * @throws ElasticException for communication problems with the server

View on GitHub (pinned to d5f144c24d)

Solutions

1. Treat as a transient consistency gap and retry the query after a short delay (replication/refresh may catch up).
2. If persistent, identify the dangling vector->meta reference and re-ingest the missing meta document, or rebuild the repository index.
3. Check ES cluster health (unassigned shards, red status) which often accompanies such gaps.

Example fix

// before
long c = db.fetchVectorCounts(it1, it2, n);
// after
try { return db.fetchVectorCounts(it1, it2, n); }
catch (ElasticException e) {
  if (e.getMessage().startsWith("meta document does not exist")) { sleep(2000); return db.fetchVectorCounts(it1copy, it2copy, n); }
  throw e;
}

// Pre-validate consistency by checking cluster health and the specific meta id presence:
public static boolean metaPresent(ElasticConnection c, String b64Id) throws ElasticException {
    JsonObject r = c.executeStatementExpectFailure(ElasticConnection.GET, "meta/_doc/" + b64Id, "");
    JsonElement found = ((JsonObject) r).get("found");
    return found != null && found.getAsString().equals("true");
}

try {
    return db.fetchVectorCounts(it1, it2, n);
} catch (ElasticException e) {
    if (!e.getMessage().startsWith("meta document does not exist")) throw e;
    Thread.sleep(2000); // allow refresh/replication to converge
    return db.fetchVectorCounts(copy(it1), copy(it2), n);
}

When it happens

Trigger: A vector result references a metaid, the meta/_mget returns a doc object with that _id but _source is null (found:false-style tombstone), so count cannot be read. Happens after a partial delete of meta docs, mid-ingest before meta is written, replication lag returning a stale tombstone, or index corruption.

Common situations: Interrupted BSim ingest that wrote vectors but not all meta docs; a delete/rollback that left dangling vector->meta references; replica inconsistency during a node failure.

Related errors

• Mismatch in metaid
• Mismatch in vectorid
• vector document does not exist for id=
• No functions matching vectorid:
• Could not recover unique executable via id

AI-assisted analysis of NationalSecurityAgency/ghidra@d5f144c24d (2026-08-14). Data as JSON: /api/errors/03851de66088f47f. Report an issue: GitHub.