{"record":{"id":"525e6e2e815067c9","repo":"quickwit-oss/tantivy","slug":"doc-encoding-failed-this-is-a-bug","errorCode":null,"errorMessage":"doc encoding failed. This is a bug","messagePattern":"doc encoding failed\\. This is a bug","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/schema/document/mod.rs","lineNumber":269,"sourceCode":"    fn to_named_doc(&self, schema: &Schema) -> NamedFieldDocument {\n        let mut field_map = BTreeMap::new();\n        for (field, field_values) in self.get_sorted_field_values() {\n            let field_name = schema.get_field_name(field);\n            let values: Vec<OwnedValue> = field_values\n                .into_iter()\n                .map(|val| OwnedValue::from(val.as_value()))\n                .collect();\n            field_map.insert(field_name.to_string(), values);\n        }\n        NamedFieldDocument(field_map)\n    }\n\n    /// Encode the doc in JSON.\n    ///\n    /// Encoding a document cannot fail.\n    fn to_json(&self, schema: &Schema) -> String {\n        serde_json::to_string(&self.to_named_doc(schema))\n            .expect(\"doc encoding failed. This is a bug\")\n    }\n}\n\npub(crate) mod type_codes {\n    pub const TEXT_CODE: u8 = 0;\n    pub const U64_CODE: u8 = 1;\n    pub const I64_CODE: u8 = 2;\n    pub const HIERARCHICAL_FACET_CODE: u8 = 3;\n    pub const BYTES_CODE: u8 = 4;\n    pub const DATE_CODE: u8 = 5;\n    pub const F64_CODE: u8 = 6;\n    pub const EXT_CODE: u8 = 7;\n\n    #[deprecated]\n    pub const JSON_OBJ_CODE: u8 = 8; // Replaced by the `OBJECT_CODE`.\n    pub const BOOL_CODE: u8 = 9;\n    pub const IP_CODE: u8 = 10;\n    pub const NULL_CODE: u8 = 11;","sourceCodeStart":251,"sourceCodeEnd":287,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/src/schema/document/mod.rs#L251-L287","documentation":"This panic occurs inside `Document::to_json`, which serializes the document's named fields via `serde_json::to_string`. The API documents that encoding a document cannot fail, so a `serde_json` error is treated as an internal bug and surfaced with `expect(\"doc encoding failed. This is a bug\")`. In practice it means serialization produced an error despite the document model being JSON-safe.","triggerScenarios":"Calling `to_json(&schema)` (directly or indirectly through document printing/debugging) when `serde_json::to_string` fails — realistically only from a poisoned/broken custom `Value` implementation that yields non-serializable data, or a memory/allocation failure inside serde_json.","commonSituations":"Custom `Document`/`Value` trait implementations that produce values serde cannot serialize (e.g. maps with non-string keys when not using serde's map-key escaping correctly), or using a forked/patched serde_json. Extremely rare for standard users.","solutions":["Audit any custom `Document` or `Value` implementation to ensure `to_named_doc` only returns `serde_json::Value`-compatible data (string keys, JSON scalar/array/object values).","Test document serialization in isolation with `serde_json::to_string(&doc.to_named_doc(schema))` and handle/display the actual error to identify the offending value.","Upgrade/align tantivy and serde_json versions if using a patched or mismatched dependency tree.","If it reproduces with stock tantivy values, file a bug with a minimal reproducing document — the library treats this path as infallible by contract."],"exampleFix":"// before: custom value with non-string map key\nNamedDoc::from(map_with_integer_keys)\n\n// after: stringify keys before handing to serde\nNamedDoc::from(map.into_iter().map(|(k, v)| (k.to_string(), v)).collect())","handlingStrategy":"try-catch","validationCode":"fn check_doc_serializable(doc: &Document, schema: &Schema) -> Result<(), serde_json::Error> {\n    serde_json::to_string(&doc.to_named_doc(schema))\n        .map(|_| ())\n        .map_err(|e| e)\n}","typeGuard":"fn is_json_safe(v: &serde_json::Value) -> bool {\n    match v {\n        serde_json::Value::Object(m) => m.iter().all(|(k, v)| !k.is_empty() && is_json_safe(v)),\n        serde_json::Value::Array(a) => a.iter().all(is_json_safe),\n        _ => true,\n    }\n}","tryCatchPattern":"let json = std::panic::catch_unwind(|| doc.to_json(schema))\n    .map_err(|_| anyhow::anyhow!(\"doc serialization failed — inspect custom Value impl\"))?;","preventionTips":["Only implement custom Document/Value types returning plain serde_json::Value data","Unit-test to_json on representative documents including edge-case custom values","Keep tantivy and serde_json versions aligned; avoid patched forks of either"],"tags":["serialization","json","panic","serde"],"backgroundTag":"json-serialization-failed","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}