{"record":{"id":"cb2c9c17737d5c99","repo":"quickwit-oss/tantivy","slug":"could-not-convert-to-string","errorCode":null,"errorMessage":"could not convert to String","messagePattern":"could not convert to String","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/aggregation/bucket/composite/collector.rs","lineNumber":452,"sourceCode":"}\n\nfn resolve_term(\n    val: u64,\n    column_type: &ColumnType,\n    str_dict_column: &Option<StrColumn>,\n    column: &Column,\n) -> crate::Result<CompositeIntermediateKey> {\n    let key = if *column_type == ColumnType::Str {\n        let fallback_dict = Dictionary::empty();\n        let term_dict = str_dict_column\n            .as_ref()\n            .map(|el| el.dictionary())\n            .unwrap_or_else(|| &fallback_dict);\n\n        let mut buffer = Vec::new();\n        term_dict.ord_to_term(val, &mut buffer)?;\n        CompositeIntermediateKey::Str(\n            String::from_utf8(buffer.to_vec()).expect(\"could not convert to String\"),\n        )\n    } else if *column_type == ColumnType::DateTime {\n        let val = i64::from_u64(val);\n        CompositeIntermediateKey::DateTime(val)\n    } else if *column_type == ColumnType::Bool {\n        let val = bool::from_u64(val);\n        CompositeIntermediateKey::Bool(val)\n    } else if *column_type == ColumnType::IpAddr {\n        let compact_space_accessor = column\n            .values\n            .clone()\n            .downcast_arc::<CompactSpaceU64Accessor>()\n            .map_err(|_| {\n                TantivyError::AggregationError(crate::aggregation::AggregationError::InternalError(\n                    \"Type mismatch: Could not downcast to CompactSpaceU64Accessor\".to_string(),\n                ))\n            })?;\n        let val: u128 = compact_space_accessor.compact_to_u128(val as u32);","sourceCodeStart":434,"sourceCodeEnd":470,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/src/aggregation/bucket/composite/collector.rs#L434-L470","documentation":"resolve_term converts a term ordinal back to bytes and then, for Str columns, .expect()s that the bytes are valid UTF-8, panicking with \"could not convert to String\" if not. The term dictionary is assumed to hold well-formed UTF-8 strings; invalid bytes imply corruption of the dictionary or that the value actually belongs to a non-str column type.","triggerScenarios":"Calling resolve_term (via resolve_internal_value_repr) on a Str-typed column whose ord_to_term output is not valid UTF-8: corrupted term dictionary bytes, wrong column type classification (binary data stored as str), or val pointing at an out-of-range ordinal producing garbage.","commonSituations":"Indexes written by third-party/older tools with non-UTF-8 term bytes; user data injected as raw bytes into a str field; segment corruption after disk issues.","solutions":["Fix the source data/indexing so str fields contain valid UTF-8; re-index the affected segment.","Replace the expect with String::from_utf8_lossy (or return TantivyError::InternalError) to degrade gracefully.","Verify the column_type check before this branch — a Bytes column misclassified as Str would land here.","Validate the segment/dictionary integrity if corruption is suspected."],"exampleFix":"// before\nString::from_utf8(buffer.to_vec()).expect(\"could not convert to String\")\n// after\nString::from_utf8_lossy(&buffer).into_owned()","handlingStrategy":"validation","validationCode":"// Validate UTF-8 before conversion\nmatch std::str::from_utf8(&buffer) {\n    Ok(s) => CompositeIntermediateKey::Str(s.to_string()),\n    Err(e) => return Err(TantivyError::InternalError(format!(\n        \"term is not valid UTF-8: {}\", e))),\n}","typeGuard":"fn valid_utf8_term(bytes: &[u8]) -> Option<&str> {\n    std::str::from_utf8(bytes).ok()\n}","tryCatchPattern":null,"preventionTips":["Never index non-UTF-8 bytes into str fields","Use from_utf8_lossy for defensive reads of dictionaries","Verify column type before treating values as strings","Check segment integrity when unexpected byte garbage appears"],"tags":["rust","panic","aggregation","utf-8","composite"],"backgroundTag":"invalid-utf8-term-bytes","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}