dbt-labs/dbt-core · error

lowercase_column_names: schema and columns should be compati

Error message

lowercase_column_names: schema and columns should be compatible

What it means

lowercase_column_names rebuilds the RecordBatch with a schema whose field names are lowercased but keeps the original column arrays untouched. Since only names change (not types or lengths), RecordBatch::try_new should always succeed; an Err means the batch was already corrupt (column count/length mismatch vs schema), which is an internal invariant violation and panics via .expect. It is invoked from normalize_result_column_names when normalizing driver result metadata.

Source

Thrown at crates/dbt-adapter/src/record_batch.rs:245

            f.name()
                .chars()
                .all(|c| c.is_lowercase() || !c.is_alphabetic())
        }) {
            return self;
        }

        let new_fields: Vec<_> = fields
            .iter()
            .map(|f| Arc::new(f.as_ref().clone().with_name(f.name().to_lowercase())))
            .collect();

        let new_schema = Arc::new(Schema::new_with_metadata(
            new_fields,
            schema.metadata().clone(),
        ));

        RecordBatch::try_new(new_schema, self.columns().to_vec())
            .expect("lowercase_column_names: schema and columns should be compatible")
    }
}

pub trait StructArrayExt {
    /// Looks up a named field in the struct and returns it as a typed array `T`.
    /// Errors if the field is absent or is not of type `T`.
    fn column_as<T: 'static>(&self, name: &str) -> AdapterResult<&T>;
}

impl StructArrayExt for StructArray {
    fn column_as<T: 'static>(&self, name: &str) -> AdapterResult<&T> {
        self.column_by_name(name)
            .and_then(|c| c.as_any().downcast_ref::<T>())
            .ok_or_else(|| {
                AdapterError::new(
                    AdapterErrorKind::UnexpectedResult,
                    format!("Missing or invalid '{name}' column"),
                )

View on GitHub (pinned to 0267ce9170)

Solutions

1. Validate the incoming batch (schema.fields().len() == batch.num_columns() and each array len == batch.num_rows()) before normalization.
2. Fix the upstream code/driver that produced the inconsistent RecordBatch; the lowercasing path is not the root cause.
3. Pin/upgrade arrow-rs to the workspace-pinned version to rule out version-specific behavior.
4. Report the offending query/driver to the adapter maintainers, since the message marks this as a should-never-happen path.

Example fix

// before
RecordBatch::try_new(new_schema, self.columns().to_vec())
    .expect("lowercase_column_names: schema and columns should be compatible")
// after
RecordBatch::try_new(new_schema, self.columns().to_vec()).map_err(|e| {
    AdapterError::new(AdapterErrorKind::Internal, format!(
        "lowercase_column_names: incompatible batch: {e}"))
})?

fn safe_lowercase(batch: RecordBatch) -> Option<RecordBatch> {
    let schema = batch.schema();
    let ok = schema.fields().len() == batch.num_columns()
        && schema.fields().iter().zip(batch.columns()).all(|(f, c)| c.len() == batch.num_rows());
    ok.then_some(batch)
}

When it happens

Trigger: Calling lowercase_column_names() (directly or through normalize_result_column_names) on a RecordBatch whose columns do not already match its own schema in count or length — the lowercasing itself never introduces the mismatch.

Common situations: A buggy or forked driver that constructs a RecordBatch with mismatched arrays; code that mutated batch columns without rebuilding the schema; arrow version upgrades changing try_new strictness.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• jsonify_nested_columns: rewritten schema and columns are con
• expected_field should be Struct
• AgateTable exists
• make_encoder for nested column should not fail
• arrow_json::Encoder emits UTF-8

AI-assisted analysis of dbt-labs/dbt-core@0267ce9170 (2026-09-07). Data as JSON: /api/errors/391746f31fbc48d5. Report an issue: GitHub.