quickwit-oss/quickwit · error

expected StringArray for service col page

Error message

expected StringArray for service col page

What it means

collect_service_names_from_page inspects the runtime DataType of the service column: for Utf8 it downcasts to arrow StringArray, for LargeUtf8 to LargeStringArray. A failure of the downcast after a matching data_type() check would violate Arrow's invariants, so it is defended with this error — indicating an internal inconsistency rather than user error.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/streaming/body_assembler.rs:115

    out: &mut HashSet<String>,
) -> Result<()> {
    use arrow::array::AsArray;
    use arrow::datatypes::{Int8Type, Int16Type, Int32Type, Int64Type};

    fn extend_from_strings(strings: &arrow::array::StringArray, out: &mut HashSet<String>) {
        for i in 0..strings.len() {
            if strings.is_valid(i) {
                out.insert(strings.value(i).to_string());
            }
        }
    }

    match arr.data_type() {
        DataType::Utf8 => {
            let strings = arr
                .as_any()
                .downcast_ref::<arrow::array::StringArray>()
                .ok_or_else(|| anyhow!("expected StringArray for service col page"))?;
            extend_from_strings(strings, out);
        }
        DataType::LargeUtf8 => {
            let strings = arr
                .as_any()
                .downcast_ref::<arrow::array::LargeStringArray>()
                .ok_or_else(|| anyhow!("expected LargeStringArray for service col page"))?;
            for i in 0..strings.len() {
                if strings.is_valid(i) {
                    out.insert(strings.value(i).to_string());
                }
            }
        }
        DataType::Dictionary(key_type, value_type)
            if matches!(value_type.as_ref(), DataType::Utf8) =>
        {
            // Extract the dictionary's values that are referenced by
            // valid (non-null) keys.

View on GitHub (pinned to a39730c5cd)

Solutions

1. Report/investigate as a bug: the data type tag disagrees with the concrete array type.
2. Verify the input parquet file's service column is a valid Utf8/LargeUtf8 array and not corrupted (re-read the file with arrow readers).
3. Check the arrow version for known downcast bugs and align all crates on one arrow version.

// before consuming the service column
debug_assert_eq!(arr.data_type(), &DataType::Utf8);
debug_assert!(arr.as_any().downcast_ref::<arrow::array::StringArray>().is_some());

fn as_string_array(arr: &dyn arrow::array::Array) -> Option<&arrow::array::StringArray> {
    if arr.data_type() == &DataType::Utf8 {
        arr.as_any().downcast_ref::<arrow::array::StringArray>()
    } else {
        None
    }
}

// error indicates an internal invariant violation; log with context and abort
if let Err(e) = iterator.next() {
    tracing::error!(error = %e, "arrow array type tag mismatch in service column");
    return Err(e);
}

When it happens

Trigger: Iterating a page of the service column whose DataType reports Utf8 but whose underlying buffer is not a StringArray — essentially only reachable through an internal bug, corrupted column data, or a custom arrow implementation.

Common situations: Bugs in column assembly code that mislabels the data type; corrupted or manually crafted arrow arrays passed into the streaming merge iterator.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• expected LargeStringArray for service col page
• timestamp_secs must be UInt64 or Int64 for MC-3 check
• sort schema '{}' does not contain a timestamp column
• input {} is missing column '{}'
• `{SORTED_SERIES_COLUMN}` must be Binary-typed

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/b85de612114fa6cd. Report an issue: GitHub.