NationalSecurityAgency/ghidra · error · IOException

Document is missing "features"

Error message

Document is missing "features"

What it means

VectorCompareScript reads each stored document's source expecting a field whose key is FEATURES_NAME ("features"). scanForFeatures walks the byte buffer skipping spaces/tabs while matching the literal key; any non-matching, non-whitespace byte means the field is absent or misplaced, so IOException is thrown.

Solutions

1. Ensure indexed documents carry the "features" field exactly as produced by the BSim indexer.
2. Re-index the affected documents with the correct mapping/analyzer.
3. Verify the analysis plugin version and document format match what the query expects.

Example fix

// before: document source lacks the features field
{ "name": "fn", "sig": "..." }
// after: include the features field produced by the BSim indexer
{ "name": "fn", "features": "<base64-vector>", "sig": "..." }

// Before querying, confirm stored documents contain the features field:
// GET /<index>/_count?q=!_exists_:features
// Re-index any documents that fail this filter.

When it happens

Trigger: A stored document whose source does not begin (after whitespace) with the expected "features" key - schema drift, a missing field, or a field under a different name.

Common situations: Indexing documents without the BSim features field; a mismatched mapping/analyzer between the index and the plugin; documents written by a different/older indexer; partial or hand-edited documents.

Related errors

• Document does not contain complete "features"
• Bad encoding in result document
• Cluster did not accept settings for index:
• Could not find function
• Could not find hits document

AI-assisted analysis of NationalSecurityAgency/ghidra@d5f144c24d (2026-08-14). Data as JSON: /api/errors/1d75e867a6215a2a. Report an issue: GitHub.

Appendix: source

Thrown at Ghidra/Extensions/BSimElasticPlugin/src/org/elasticsearch/plugin/analysis/lsh/VectorCompareScriptFactory.java:94

		@Override
		public boolean needs_termStats() {
			return false;
		}

		private static int scanForFeatures(byte[] buffer, int offset) throws IOException {
			int i = 0;
			while (i < FEATURES_NAME.length()) {
				char curChar = FEATURES_NAME.charAt(i);
				int val = buffer[offset];
				if (val == curChar) {
					i += 1;
					offset += 1;
				}
				else if (val == ' ' || val == '\t') {
					offset += 1;
				}
				else {
					throw new IOException("Document is missing \"features\"");
				}
			}
			return offset;
		}

		private static int scanForLength(BytesRef byteRef, int startOffset) throws IOException {
			int finalLength = 0;
			int maxLength = byteRef.length - (startOffset - byteRef.offset);
			while (finalLength < maxLength) {
				if (byteRef.bytes[finalLength + startOffset] == '\"') {
					break;
				}
				finalLength += 1;
			}
			if (finalLength == byteRef.length) {
				throw new IOException("Document does not contain complete \"features\"");
			}
			return finalLength;

View on GitHub (pinned to d5f144c24d)