{"record":{"id":"1d75e867a6215a2a","repo":"NationalSecurityAgency/ghidra","slug":"document-is-missing-features","errorCode":null,"errorMessage":"Document is missing \"features\"","messagePattern":"Document is missing \"features\"","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"Ghidra/Extensions/BSimElasticPlugin/src/org/elasticsearch/plugin/analysis/lsh/VectorCompareScriptFactory.java","lineNumber":94,"sourceCode":"\t\t@Override\n\t\tpublic boolean needs_termStats() {\n\t\t\treturn false;\n\t\t}\n\n\t\tprivate static int scanForFeatures(byte[] buffer, int offset) throws IOException {\n\t\t\tint i = 0;\n\t\t\twhile (i < FEATURES_NAME.length()) {\n\t\t\t\tchar curChar = FEATURES_NAME.charAt(i);\n\t\t\t\tint val = buffer[offset];\n\t\t\t\tif (val == curChar) {\n\t\t\t\t\ti += 1;\n\t\t\t\t\toffset += 1;\n\t\t\t\t}\n\t\t\t\telse if (val == ' ' || val == '\\t') {\n\t\t\t\t\toffset += 1;\n\t\t\t\t}\n\t\t\t\telse {\n\t\t\t\t\tthrow new IOException(\"Document is missing \\\"features\\\"\");\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn offset;\n\t\t}\n\n\t\tprivate static int scanForLength(BytesRef byteRef, int startOffset) throws IOException {\n\t\t\tint finalLength = 0;\n\t\t\tint maxLength = byteRef.length - (startOffset - byteRef.offset);\n\t\t\twhile (finalLength < maxLength) {\n\t\t\t\tif (byteRef.bytes[finalLength + startOffset] == '\\\"') {\n\t\t\t\t\tbreak;\n\t\t\t\t}\n\t\t\t\tfinalLength += 1;\n\t\t\t}\n\t\t\tif (finalLength == byteRef.length) {\n\t\t\t\tthrow new IOException(\"Document does not contain complete \\\"features\\\"\");\n\t\t\t}\n\t\t\treturn finalLength;","sourceCodeStart":76,"sourceCodeEnd":112,"githubUrl":"https://github.com/NationalSecurityAgency/ghidra/blob/d5f144c24d6bc53c9cbf4448c6d11143e7696206/Ghidra/Extensions/BSimElasticPlugin/src/org/elasticsearch/plugin/analysis/lsh/VectorCompareScriptFactory.java#L76-L112","documentation":"VectorCompareScript reads each stored document's source expecting a field whose key is FEATURES_NAME (\"features\"). scanForFeatures walks the byte buffer skipping spaces/tabs while matching the literal key; any non-matching, non-whitespace byte means the field is absent or misplaced, so IOException is thrown.","triggerScenarios":"A stored document whose source does not begin (after whitespace) with the expected \"features\" key - schema drift, a missing field, or a field under a different name.","commonSituations":"Indexing documents without the BSim features field; a mismatched mapping/analyzer between the index and the plugin; documents written by a different/older indexer; partial or hand-edited documents.","solutions":["Ensure indexed documents carry the \"features\" field exactly as produced by the BSim indexer.","Re-index the affected documents with the correct mapping/analyzer.","Verify the analysis plugin version and document format match what the query expects."],"exampleFix":"// before: document source lacks the features field\n{ \"name\": \"fn\", \"sig\": \"...\" }\n// after: include the features field produced by the BSim indexer\n{ \"name\": \"fn\", \"features\": \"<base64-vector>\", \"sig\": \"...\" }","handlingStrategy":"validation","validationCode":"// Before querying, confirm stored documents contain the features field:\n// GET /<index>/_count?q=!_exists_:features\n// Re-index any documents that fail this filter.","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Index documents only via the BSim indexer so the features field is always present.","Validate the mapping/analyzer matches the plugin version before querying."],"tags":["elasticsearch","bsim","indexing","data-format"],"backgroundTag":null,"analyzedSha":"d5f144c24d6bc53c9cbf4448c6d11143e7696206","analyzedAt":"2026-08-14T01:00:57.564Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}