NationalSecurityAgency/ghidra · error · IOException

Document is missing "features"

Error message

Document is missing "features"

What it means

VectorCompareScript reads each stored document's source expecting a field whose key is FEATURES_NAME ("features"). scanForFeatures walks the byte buffer skipping spaces/tabs while matching the literal key; any non-matching, non-whitespace byte means the field is absent or misplaced, so IOException is thrown.

Source

Thrown at Ghidra/Extensions/BSimElasticPlugin/src/org/elasticsearch/plugin/analysis/lsh/VectorCompareScriptFactory.java:94

		@Override
		public boolean needs_termStats() {
			return false;
		}

		private static int scanForFeatures(byte[] buffer, int offset) throws IOException {
			int i = 0;
			while (i < FEATURES_NAME.length()) {
				char curChar = FEATURES_NAME.charAt(i);
				int val = buffer[offset];
				if (val == curChar) {
					i += 1;
					offset += 1;
				}
				else if (val == ' ' || val == '\t') {
					offset += 1;
				}
				else {
					throw new IOException("Document is missing \"features\"");
				}
			}
			return offset;
		}

		private static int scanForLength(BytesRef byteRef, int startOffset) throws IOException {
			int finalLength = 0;
			int maxLength = byteRef.length - (startOffset - byteRef.offset);
			while (finalLength < maxLength) {
				if (byteRef.bytes[finalLength + startOffset] == '\"') {
					break;
				}
				finalLength += 1;
			}
			if (finalLength == byteRef.length) {
				throw new IOException("Document does not contain complete \"features\"");
			}
			return finalLength;

View on GitHub (pinned to d5f144c24d)

Solutions

1. Ensure indexed documents carry the "features" field exactly as produced by the BSim indexer.
2. Re-index the affected documents with the correct mapping/analyzer.
3. Verify the analysis plugin version and document format match what the query expects.

Example fix

// before: document source lacks the features field
{ "name": "fn", "sig": "..." }
// after: include the features field produced by the BSim indexer
{ "name": "fn", "features": "<base64-vector>", "sig": "..." }

// Before querying, confirm stored documents contain the features field:
// GET /<index>/_count?q=!_exists_:features
// Re-index any documents that fail this filter.

When it happens

Trigger: A stored document whose source does not begin (after whitespace) with the expected "features" key - schema drift, a missing field, or a field under a different name.

Common situations: Indexing documents without the BSim features field; a mismatched mapping/analyzer between the index and the plugin; documents written by a different/older indexer; partial or hand-edited documents.

Related errors

• Document does not contain complete "features"
• {getType()}scripts cannot be used for context [{context.name
• Unknown script name {scriptSource}
• Unknown error format
• Mismatch in metaid

AI-assisted analysis of NationalSecurityAgency/ghidra@d5f144c24d (2026-08-14). Data as JSON: /api/errors/1d75e867a6215a2a. Report an issue: GitHub.