{"record":{"id":"062fe15eaffd7323","repo":"quickwit-oss/tantivy","slug":"invalid-type-code-in-json-term","errorCode":null,"errorMessage":"Invalid type code in JSON term","messagePattern":"Invalid type code in JSON term","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"src/postings/json_postings_writer.rs","lineNumber":84,"sourceCode":"        doc_id_map: Option<&DocIdMapping>,\n        ctx: &IndexingContext,\n        serializer: &mut FieldSerializer,\n    ) -> io::Result<()> {\n        let mut term_buffer = JsonTermSerializer(Vec::with_capacity(48));\n        let mut buffer_lender = BufferLender::default();\n        let mut prev_term_id = u32::MAX;\n        let mut term_path_len = 0; // this will be set in the first iteration\n        for (_field, path_id, term, addr) in ordered_term_addrs {\n            if prev_term_id != path_id.path_id() {\n                term_buffer.clear();\n                term_buffer.append_json_path(ordered_id_to_path[path_id.path_id() as usize]);\n                term_path_len = term_buffer.len();\n                prev_term_id = path_id.path_id();\n            }\n            term_buffer.truncate(term_path_len);\n            term_buffer.append_bytes(term);\n\n            let typ = Type::from_code(term[0]).expect(\"Invalid type code in JSON term\");\n            if typ == Type::Str {\n                SpecializedPostingsWriter::<Rec>::serialize_one_term(\n                    term_buffer.as_bytes(),\n                    *addr,\n                    doc_id_map,\n                    &mut buffer_lender,\n                    ctx,\n                    serializer,\n                )?;\n            } else {\n                SpecializedPostingsWriter::<DocIdRecorder>::serialize_one_term(\n                    term_buffer.as_bytes(),\n                    *addr,\n                    doc_id_map,\n                    &mut buffer_lender,\n                    ctx,\n                    serializer,\n                )?;","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/src/postings/json_postings_writer.rs#L66-L102","documentation":"During JSON postings serialization, the writer reads term[0] as a type code and maps it via Type::from_code, panicking with \"Invalid type code in JSON term\" if the first byte is not a known Type code (src/postings/json_postings_writer.rs:84). JSON terms in tantivy are encoded with a type prefix byte (Str/Date/U64/I64/F64/Bool), so a corrupted or wrongly-prefixed term buffer breaks this invariant. This is an internal-consistency panic, normally unreachable from public APIs.","triggerScenarios":"Indexing or merging JSON documents whose term dictionary produced a term whose first byte is not a valid type code; typically caused by a library bug, corrupted segment files during serialization/merge, or manually crafted terms inserted via low-level APIs.","commonSituations":"Segment corruption from truncated writes or bit-rot on disk; mixing segment files across incompatible tantivy versions; custom code building JSON-path terms with hand-set bytes; hitting a genuine serializer bug during merge of JSON fields.","solutions":["Delete and rebuild the index — corrupted segments are the most common cause and cannot be repaired in place.","Ensure all segments were written by the same tantivy version; do not mix data directories across versions.","If you build JSON terms manually, make the first byte a valid Type code matching the encoded value.","Reduce to a reproducible input document and file a bug with the tantivy maintainers if the index is freshly built."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"// verify segment files were all written by the current tantivy version\n// and pass integrity checks before search/merge","typeGuard":null,"tryCatchPattern":"let ok = std::panic::catch_unwind(AssertUnwindSafe(|| merge_or_search()));\nif ok.is_err() { rebuild_index_from_source()?; }","preventionTips":["Rebuild the index if you ever see this — assume corruption","Never mix segment directories across tantivy versions","Don't hand-craft JSON term bytes; use Term builder APIs","Use durable writes/fsync to avoid truncated segments"],"tags":["rust","panic","json","postings","corruption","indexing"],"backgroundTag":"invalid-type-code","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}