{"record":{"id":"30f1610b0eb40a18","repo":"quickwit-oss/quickwit","slug":"dictionary-downcast-failed-for","errorCode":null,"errorMessage":"dictionary downcast failed for {:?}","messagePattern":"dictionary downcast failed for (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/sorted_series/mod.rs","lineNumber":327,"sourceCode":"}\n\n/// Extract a string value from a column at the given row.\n///\n/// Supports `Dictionary(Int32, Utf8)` (the common tag encoding) and\n/// plain `Utf8` columns. Returns an error for unsupported types — tag\n/// columns in the sort schema must be string-typed.\nfn extract_string_value(array: &dyn Array, row: usize) -> Result<&str> {\n    debug_assert!(\n        !array.is_null(row),\n        \"caller must check is_null before extract_string_value\"\n    );\n\n    match array.data_type() {\n        DataType::Dictionary(_, _) => {\n            let dict = array\n                .as_any()\n                .downcast_ref::<DictionaryArray<Int32Type>>()\n                .ok_or_else(|| anyhow!(\"dictionary downcast failed for {:?}\", array.data_type()))?;\n            let key_idx = dict.keys().value(row) as usize;\n            let values = dict.values();\n            let str_values = values\n                .as_any()\n                .downcast_ref::<StringArray>()\n                .ok_or_else(|| anyhow!(\"dictionary values are not Utf8\"))?;\n            Ok(str_values.value(key_idx))\n        }\n        DataType::Utf8 => {\n            let arr = array\n                .as_any()\n                .downcast_ref::<StringArray>()\n                .ok_or_else(|| anyhow!(\"Utf8 downcast failed\"))?;\n            Ok(arr.value(row))\n        }\n        other => Err(anyhow!(\n            \"unsupported data type {:?} in sort schema tag column — only Dictionary(Int32, Utf8) \\\n             and Utf8 are supported\",","sourceCodeStart":309,"sourceCodeEnd":345,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/sorted_series/mod.rs#L309-L345","documentation":"extract_string_value supports only Dictionary(Int32, Utf8) arrays for string tag columns. When the Arrow array's data type is a dictionary, it attempts a downcast to DictionaryArray<Int32Type>; if the keys are not Int32 (e.g. UInt8/UInt16 dictionary from a different parquet writer), the downcast returns None and this error is raised.","triggerScenarios":"encode_row_key hits a tag column whose Arrow array is DataType::Dictionary(_, _) but with a non-Int32 key type — downcast_ref::<DictionaryArray<Int32Type>>() returns None.","commonSituations":"Reading parquet files written with dictionary-encoded strings using smaller key widths (UInt8/UInt16/UInt64), or files produced by tools that pick a different dictionary index type based on cardinality.","solutions":["Check the parquet writer settings that choose the dictionary index type and force Int32 keys","Normalize the array before calling (e.g. cast dictionary keys to Int32 via arrow::compute::cast)","Widen extract_string_value to handle other dictionary key widths","Inspect array.data_type() in the error message to see the actual key type"],"exampleFix":"// before\nlet dict = array.as_any().downcast_ref::<DictionaryArray<Int32Type>>().ok_or_else(...)?;\n// after: normalize non-Int32 dictionaries first\nlet array = match array.data_type() {\n    DataType::Dictionary(k, _) if **k != DataType::Int32 => arrow::compute::cast(array, &DataType::Dictionary(DataType::Int32.into(), DataType::Utf8.into()))?,\n    _ => array.clone(),\n};","handlingStrategy":"type-guard","validationCode":"if let DataType::Dictionary(k, v) = array.data_type() {\n    if **k != DataType::Int32 || **v != DataType::Utf8 { /* normalize via cast */ }\n}","typeGuard":"fn is_int32_utf8_dict(array: &dyn Array) -> bool {\n    matches!(array.data_type(), DataType::Dictionary(k, v) if **k == DataType::Int32 && **v == DataType::Utf8)\n}","tryCatchPattern":"match encode_row_key(...) {\n    Err(e) if e.to_string().contains(\"dictionary downcast failed\") => {\n        let arr = arrow::compute::cast(array, &DataType::Utf8)?; /* retry */\n    }\n    Err(e) => return Err(e),\n    Ok(v) => v,\n}","preventionTips":["Normalize dictionaries to (Int32, Utf8) right after parquet read","Configure the parquet reader/writer to use consistent dictionary index types","Assert tag column types when loading batches"],"tags":["arrow","parquet","downcast","dictionary-encoding"],"backgroundTag":"type-mismatch","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}