{"record":{"id":"ba496844d4081eff","repo":"quickwit-oss/tantivy","slug":"could-not-convert-to-string-ba4968","errorCode":null,"errorMessage":"could not convert to String","messagePattern":"could not convert to String","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/aggregation/bucket/term_agg/mod.rs","lineNumber":1372,"sourceCode":"\n            let intermediate_entries: Vec<IntermediateTermBucketEntry> = buckets\n                .into_iter()\n                .map(|bucket| {\n                    into_intermediate_bucket_entry(\n                        bucket,\n                        reborrow_opt_collector(&mut sub_agg_collector),\n                        agg_data,\n                    )\n                })\n                .collect::<crate::Result<_>>()?;\n\n            let mut intermediate_entry_it = intermediate_entries.into_iter();\n\n            term_dict.sorted_ords_to_term_cb(&term_ids[..], |term| {\n                let intermediate_entry = intermediate_entry_it.next().unwrap();\n                dict.insert(\n                    IntermediateKey::Str(\n                        String::from_utf8(term.to_vec()).expect(\"could not convert to String\"),\n                    ),\n                    intermediate_entry,\n                );\n            })?;\n\n            if term_req.req.min_doc_count == 0 {\n                // TODO: Handle rev streaming for descending sorting by keys\n                let mut stream = term_dict.stream()?;\n                let empty_sub_aggregation =\n                    IntermediateAggregationResults::empty_from_req(&term_req.sug_aggregations);\n                while stream.advance() {\n                    if dict.len() >= term_req.req.segment_size as usize {\n                        break;\n                    }\n\n                    // Respect allowed filters if present\n                    if let Some(allowed_bs) = term_req.allowed_term_ids.as_ref() {\n                        if !allowed_bs.contains(stream.term_ord() as u32) {","sourceCodeStart":1354,"sourceCodeEnd":1390,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/src/aggregation/bucket/term_agg/mod.rs#L1354-L1390","documentation":"A panic from `String::from_utf8(term.to_vec()).expect(\"could not convert to String\")` in the terms aggregation finalization. `sorted_ords_to_term_cb` yields raw term bytes from the term dictionary, and the code asserts they are valid UTF-8 before using them as bucket keys. The panic fires when the dictionary returns non-UTF-8 bytes, i.e. the index's text column is corrupt or was written with a non-UTF-8 codec.","triggerScenarios":"Finalizing a terms aggregation where `term_dict.sorted_ords_to_term_cb` invokes the callback with bytes that fail UTF-8 validation — corrupted segment, wrong column type, or index written by an incompatible writer.","commonSituations":"Torn or manually-copied index directories; ingesting binary data into a keyword field; reading old index formats after a version upgrade.","solutions":["Reindex or repair the affected index/segment so the term dictionary contains valid UTF-8 terms.","Verify the field's storage type is text/keyword and that no binary bytes are being indexed.","Check writer/reader version compatibility for the index format being read.","If you maintain the code, use String::from_utf8_lossy or propagate an error instead of panicking."],"exampleFix":"// before\nString::from_utf8(term.to_vec()).expect(\"could not convert to String\")\n// after\nString::from_utf8(term.to_vec())\n    .map_err(|_| crate::AggregationError::Internal(\"non-UTF-8 term\".to_string()))?","handlingStrategy":"validation","validationCode":"// Verify terms are valid UTF-8 before building bucket keys.\nfn term_is_utf8(term: &[u8]) -> bool {\n    std::str::from_utf8(term).is_ok()\n}","typeGuard":"fn is_utf8_term(b: &[u8]) -> bool { std::str::from_utf8(b).is_ok() }","tryCatchPattern":"let res = std::panic::catch_unwind(|| run_terms_agg(...));\nif res.is_err() {\n    // trigger index repair / reindex of the affected field\n    schedule_reindex(field);\n}","preventionTips":["Reject or sanitize binary payloads before indexing into keyword/text fields.","Validate segment checksums after copying or restoring index directories.","Reindex segments written by older engine versions before running aggregations."],"tags":["rust","panic","utf-8","aggregation","index-corruption"],"backgroundTag":"invalid-utf8-decode","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}