risingwavelabs/risingwave · error

Duplicate column `{column}` in order_key

Error message

Duplicate column `{column}` in order_key

What it means

Each column may appear at most once in order_key. The parser tracks columns case-insensitively in a HashSet (seen_columns.insert with to_ascii_lowercase) and throws when a second item resolves to an already-seen column, since duplicate sort keys are ambiguous and invalid for an Iceberg sort spec.

Source

Thrown at src/connector/src/sink/iceberg/create_table.rs:507

        let tokens = item.split_whitespace().collect_vec();
        if tokens.is_empty() {
            bail!("Invalid order key item `{item}`");
        }
        if tokens.len() > 4 {
            bail!(
                "Invalid order key item `{item}`\nHINT: Supported format is `column [asc|desc] [nulls first|last]`"
            );
        }

        let column = tokens[0];
        if !ORDER_KEY_COLUMN_RE.is_match(column) {
            bail!(
                "Invalid order key column `{column}`\nHINT: Only plain column names are supported in order_key"
            );
        }
        if !seen_columns.insert(column.to_ascii_lowercase()) {
            bail!("Duplicate column `{column}` in order_key");
        }

        let mut direction = SortDirection::Ascending;
        let mut null_order = None;
        let mut idx = 1;
        while idx < tokens.len() {
            match tokens[idx].to_ascii_lowercase().as_str() {
                "asc" => {
                    direction = SortDirection::Ascending;
                    idx += 1;
                }
                "desc" => {
                    direction = SortDirection::Descending;
                    idx += 1;
                }
                "nulls" => {
                    let order = tokens.get(idx + 1).ok_or_else(|| {
                        anyhow!(

View on GitHub (pinned to 6469eb736d)

Solutions

1. Deduplicate the column list (case-insensitively) before constructing order_key
2. If you intended different sort directions, keep only one item per column — the last duplicate does not override the first
3. Check for case-insensitive repeats like `id, ID`; identifiers are matched lowercased
4. If deduplicating programmatically, e.g. `cols.into_iter().collect::<HashSet<_>>()` on lowercased names, rebuild the string from the unique set

Example fix

// before
order_key = 'user_id asc, user_id desc'
// after
order_key = 'user_id desc'

fn order_key_has_duplicates(expr: &str) -> bool {
    let mut seen = std::collections::HashSet::new();
    expr.split(',').any(|item| {
        let col = item.trim().split_whitespace().next().unwrap_or("").to_ascii_lowercase();
        !seen.insert(col)
    })
}

fn dedup_order_key(expr: &str) -> String {
    let mut seen = std::collections::HashSet::new();
    expr.split(',')
        .filter(|item| seen.insert(item.trim().split_whitespace().next().unwrap_or("").to_ascii_lowercase()))
        .collect::<Vec<_>>()
        .join(",")
}

When it happens

Trigger: order_key strings like `id, ID`, `Name, name desc`, or literal repeats such as `created_at asc, created_at desc` reaching validate_order_key_columns.

Common situations: Accidental duplication when merging config fragments; case differences (`ID` vs `id`) masking a repeat to the eye; a generated list built from two sources that both include the same column.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• 'copy-on-write' mode is not supported for append-only iceber
• Invalid order key: empty item in `{expr}`
• Invalid order key item `{item}`
• `iceberg_compaction_config_refresh_interval_sec` must be gre
• unrecognized configs: {:?}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/3266bbcacbdb08f0. Report an issue: GitHub.