dbt-labs/dbt-core · error

should split on top-level UNION ALL

Error message

should split on top-level UNION ALL

What it means

Test panic from Option::expect at crates/dbt-adapter/src/sql/diff.rs:3777: split_union_all_top_level(sql) returned None, i.e. the splitter did not recognize a top-level UNION ALL in the input and could not produce the parts vector. The scanner must locate UNION ALL outside parentheses, strings, and comments while advancing safely over multi-byte UTF-8 characters.

Source

Thrown at crates/dbt-adapter/src/sql/diff.rs:3777

        compare_sql(sql_fusion, sql_recorded, AdapterType::Snowflake)
            .expect("Forward-fill projection column order drift should be ignored");
    }

    #[test]
    fn test_split_union_top_level_splits_and_handles_unicode() {
        // Regression test: previously this could panic if the scan index landed in the middle
        // of a multi-byte UTF-8 char (e.g. “).
        let sql = "select 1 as a /* “unicode” */ UNION      select 2 as b";
        let parts = split_union_top_level(sql).expect("should split on top-level UNION");
        assert_eq!(parts, vec!["select 1 as a", "select 2 as b"]);
    }

    #[test]
    fn test_split_union_all_top_level_splits_and_handles_unicode() {
        // Regression test: previously this could panic if the scan index landed in the middle
        // of a multi-byte UTF-8 char (e.g. “).
        let sql = "select 1 as a /* “unicode” */ UNION   ALL   select 2 as b";
        let parts = split_union_all_top_level(sql).expect("should split on top-level UNION ALL");
        assert_eq!(parts, vec!["select 1 as a", "select 2 as b"]);
    }

    #[test]
    fn test_split_union_all_top_level_does_not_split_inside_parentheses() {
        let sql = "select 1 as a union all select (select 2 as b union all select 3 as c)";
        let parts =
            split_union_all_top_level(sql).expect("should split on the top-level UNION ALL");
        assert_eq!(
            parts,
            vec![
                "select 1 as a",
                "select (select 2 as b union all select 3 as c)"
            ]
        );
    }

    #[test]

View on GitHub (pinned to 0267ce9170)

Solutions

1. Use char_indices-based scanning to avoid multi-byte char boundary issues (the exact regression this test guards)
2. Make the UNION ALL match whitespace-tolerant (skip arbitrary whitespace between UNION and ALL)
3. Ensure comments are skipped before keyword matching so UNION ALL inside /* */ never matches and top-level ones always do
4. Confirm the caller (compare_sql_structurally, diff.rs:1418) surfaces the None as a clean error rather than masking it

Example fix

// before: rigid match
if &s[i..i+9] == "UNION ALL" { ... }
// after: whitespace-tolerant, char-safe scan
let rest = skip_ws(s, after_union_keyword);
if rest.starts_with("ALL") { parts.push(...); }

fn has_top_level_union_all(sql: &str) -> bool {
    scan_top_level_ws_tolerant(sql, "UNION", "ALL")
}
if !has_top_level_union_all(sql) { eprintln!("no top-level UNION ALL; not splittable"); }

fn is_split(parts: &Option<Vec<String>>) -> bool { matches!(parts, Some(v) if v.len() > 1) }

let parts = match split_union_all_top_level(sql) {
    Some(p) => p,
    None => { vec![sql.to_string()] } // treat as single statement, don't panic
};

When it happens

Trigger: cargo test -p dbt-adapter with test_split_union_all_top_level_splits_and_handles_unicode: input "select 1 as a /* “unicode” */ UNION ALL select 2 as b" and split_union_all_top_level returns None, panicking "should split on top-level UNION ALL".

Common situations: Comment bodies with smart quotes break byte-indexed scanning; keyword matching requires exactly one space between UNION and ALL so varied whitespace fails; a regression in the shared top-level scan used by compare_sql_structurally prevents splitting comment-bearing UNION ALL queries.

Related errors

• should split on top-level UNION
• should split on the top-level UNION ALL
• bare vs wrapped CREATE VIEW body should compare as equal (or
• STRUCT field order drift should be ignored
• projection order drift should be ignored

AI-assisted analysis of dbt-labs/dbt-core@0267ce9170 (2026-09-07). Data as JSON: /api/errors/fbdcda8583406400. Report an issue: GitHub.