quickwit-oss/quickwit · error

Utf8 downcast failed

Error message

Utf8 downcast failed

What it means

For non-dictionary tag columns, extract_string_value expects DataType::Utf8 and downcasts the Arrow array to StringArray. If the downcast fails despite the data type claiming Utf8 (a broken/inconsistent array, or LargeUtf8 that slipped past the match arm), this error is raised.

Solutions

1. Check how the StringArray for this column was built — data_type() must match the actual buffers
2. Cast to Utf8 (arrow::compute::cast) before calling extract_string_value
3. Add a LargeUtf8 arm to extract_string_value if large string arrays occur in your pipeline
4. This is an invariant violation — file a bug with the array construction path if reproducible

Example fix

// before
let arr = array.as_any().downcast_ref::<StringArray>().ok_or_else(|| anyhow!("Utf8 downcast failed"))?;
// after
let arr = array.as_any().downcast_ref::<StringArray>()
    .ok_or_else(|| anyhow!("Utf8 downcast failed: declared {:?} but buffers are not StringArray (len={}, nulls={})", array.data_type(), array.len(), array.null_count()))?;

debug_assert_eq!(array.data_type(), &DataType::Utf8);
debug_assert!(array.as_any().downcast_ref::<StringArray>().is_some());

fn is_string_array(array: &dyn Array) -> bool {
    array.as_any().downcast_ref::<StringArray>().is_some()
}

if let Err(e) = encode_row_key(...) {
    if e.to_string().contains("Utf8 downcast failed") { /* invariant violation: log array metadata, abort */ }
}

When it happens

Trigger: encode_row_key hits a tag column with DataType::Utf8 whose underlying buffer cannot be downcast to StringArray — practically an internal inconsistency, or a LargeUtf8 array if the match were extended without a matching downcast.

Common situations: Arrow arrays constructed manually with mismatched data_type metadata and buffers, or LargeString arrays from other Arrow implementations.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• dictionary downcast failed for
• dictionary values are not Utf8
• unsupported data type
• append column
• expected LargeStringArray for service col page

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/10624bd4285398f4. Report an issue: GitHub.

Appendix: source

Thrown at quickwit/quickwit-parquet-engine/src/sorted_series/mod.rs:340

    match array.data_type() {
        DataType::Dictionary(_, _) => {
            let dict = array
                .as_any()
                .downcast_ref::<DictionaryArray<Int32Type>>()
                .ok_or_else(|| anyhow!("dictionary downcast failed for {:?}", array.data_type()))?;
            let key_idx = dict.keys().value(row) as usize;
            let values = dict.values();
            let str_values = values
                .as_any()
                .downcast_ref::<StringArray>()
                .ok_or_else(|| anyhow!("dictionary values are not Utf8"))?;
            Ok(str_values.value(key_idx))
        }
        DataType::Utf8 => {
            let arr = array
                .as_any()
                .downcast_ref::<StringArray>()
                .ok_or_else(|| anyhow!("Utf8 downcast failed"))?;
            Ok(arr.value(row))
        }
        other => Err(anyhow!(
            "unsupported data type {:?} in sort schema tag column — only Dictionary(Int32, Utf8) \
             and Utf8 are supported",
            other
        )),
    }
}

/// Extract an i64 value from a column at the given row.
///
/// Panics (debug_assert) if the row is null — the caller must check first.
fn extract_i64_value(array: &dyn Array, row: usize) -> i64 {
    debug_assert!(
        !array.is_null(row),
        "caller must check is_null before extract_i64_value"
    );

View on GitHub (pinned to a39730c5cd)