dbt-labs/dbt-core · error

Forward-fill projection column order drift should be ignored

Error message

Forward-fill projection column order drift should be ignored

What it means

Test panic from .expect at crates/dbt-adapter/src/sql/diff.rs:3760: compare_sql between two Snowflake forward-fill queries (CTE pipeline ending in `qualify row_number() over (partition by billing_group_id, __as_of order by rn desc) = 1`) returned Err because the projection column order of `filled_data`/final SELECT differs. The differ should treat reordered projection columns as equivalent so recorded and Fusion-generated SQL match.

Source

Thrown at crates/dbt-adapter/src/sql/diff.rs:3760

  billing_group_id,
  __as_of,
  last_value(b ignore nulls) over (
    partition by billing_group_id
    order by __as_of
    rows between unbounded preceding and current row
  ) as b,
  last_value(a ignore nulls) over (
    partition by billing_group_id
    order by __as_of
    rows between unbounded preceding and current row
  ) as a,
  rn
from filled_data
qualify row_number() over (partition by billing_group_id, __as_of order by rn desc) = 1
"#;

        compare_sql(sql_fusion, sql_recorded, AdapterType::Snowflake)
            .expect("Forward-fill projection column order drift should be ignored");
    }

    #[test]
    fn test_split_union_top_level_splits_and_handles_unicode() {
        // Regression test: previously this could panic if the scan index landed in the middle
        // of a multi-byte UTF-8 char (e.g. “).
        let sql = "select 1 as a /* “unicode” */ UNION      select 2 as b";
        let parts = split_union_top_level(sql).expect("should split on top-level UNION");
        assert_eq!(parts, vec!["select 1 as a", "select 2 as b"]);
    }

    #[test]
    fn test_split_union_all_top_level_splits_and_handles_unicode() {
        // Regression test: previously this could panic if the scan index landed in the middle
        // of a multi-byte UTF-8 char (e.g. “).
        let sql = "select 1 as a /* “unicode” */ UNION   ALL   select 2 as b";
        let parts = split_union_all_top_level(sql).expect("should split on top-level UNION ALL");
        assert_eq!(parts, vec!["select 1 as a", "select 2 as b"]);

View on GitHub (pinned to 0267ce9170)

Solutions

1. Read the Err detail to find which projection column pair failed to canonicalize to the same form
2. Ensure alias-based projection reordering is applied to the final SELECT feeding the QUALIFY clause, not just plain SELECTs
3. Check whether window-function references (rn in QUALIFY) bypassed the projection normalization
4. If the drift is real (different expression per column), fix the generator

Example fix

// before
compare_sql(sql_fusion, sql_recorded, AdapterType::Snowflake)
    .expect("Forward-fill projection column order drift should be ignored");
// after (library-side: reorder projections by alias before compare, incl. QUALIFY-fed selects)
let norm = |s: &str| sort_projections_by_alias(canonicalize(s, AdapterType::Snowflake));
assert_eq!(norm(sql_fusion), norm(sql_recorded));

let cols_a = projection_for_all(sql_fusion);
let cols_b = projection_for_all(sql_recorded);
if BTreeSet::from_iter(cols_a) != BTreeSet::from_iter(cols_b) {
    eprintln!("projection sets differ; order-insensitive compare will fail");
}

compare_sql(sql_fusion, sql_recorded, AdapterType::Snowflake)
    .unwrap_or_else(|e| eprintln!("forward-fill drift: {e}"));

When it happens

Trigger: cargo test -p dbt-adapter with test at diff.rs:3760: sql_fusion and sql_recorded differ only in the column order of the forward-fill projection, and compare_sql(sql_fusion, sql_recorded, AdapterType::Snowflake) yields Err, panicking the expect.

Common situations: The forward-fill macro/generator emits SELECT columns in a different order than the recorded snapshot; canonicalizer's projection reordering fails when columns feed a QUALIFY/window clause; replay comparisons of incremental forward-fill models flag ordering drift.

Related errors

• bare vs wrapped CREATE VIEW body should compare as equal (or
• STRUCT field order drift should be ignored
• projection order drift should be ignored
• projection order drift should be ignored even with apostroph
• should treat persisted view vs temp view as equivalent

AI-assisted analysis of dbt-labs/dbt-core@0267ce9170 (2026-09-07). Data as JSON: /api/errors/fcbe137abb11dd38. Report an issue: GitHub.