risingwavelabs/risingwave · error

failed to refresh schema because some of the columns to drop

Error message

failed to refresh schema because some of the columns to drop are referenced by a generated column "{}"

What it means

When refreshing a schema-registry-backed table's schema, dropping columns that feed a generated column makes the generated column's expression unbindable. The code matches the inner bind error message for generated columns and re-wraps it with a clearer message naming the offending generated column; otherwise the original error propagates.

Source

Thrown at src/frontend/src/handler/alter_table_with_sr.rs:86

            table_name,
            definition,
            &original_table,
            SqlColumnStrategy::Ignore,
        ))
        .await;
        match result {
            Ok((source, table, graph, job_type)) => Ok((source, table, graph, job_type)),
            Err(e) => {
                let report = e.to_report_string();
                // NOTE(yuhao): This is a workaround for reporting errors when columns to drop is referenced by generated column.
                // Finding the actual columns to drop requires generating `PbSource` from the sql definition
                // and fetching schema from schema registry, which will cause a lot of unnecessary refactor.
                // Here we match the error message to yield when failing to bind generated column exprs.
                let re = Regex::new(r#"failed to bind the expression in generated column "(.*?)""#)
                    .unwrap();
                let captures = re.captures(&report).map_err(anyhow::Error::from)?;
                if let Some(gen_col_name) = captures.and_then(|captures| captures.get(1)) {
                    Err(anyhow!(e).context(format!("failed to refresh schema because some of the columns to drop are referenced by a generated column \"{}\"",
                gen_col_name.as_str())).into())
                } else {
                    Err(e)
                }
            }
        }
    }?;
    let catalog_writer = session.catalog_writer()?;

    catalog_writer
        .replace_table(
            source.map(|x| x.to_prost()),
            table.to_prost(),
            graph,
            job_type,
        )
        .await?;

View on GitHub (pinned to 6469eb736d)

Solutions

1. Drop the generated column (or alter it to not reference the removed field) before refreshing the schema
2. Update the schema registry subject so the column remains present
3. Recreate the table with the new schema and regenerated computed columns

-- Before refresh, check generated column expressions against incoming schema fields
SELECT name, generated_expr FROM rw_catalog.rw_columns
WHERE relation_id = <table_id> AND generated_expr IS NOT NULL;

When it happens

Trigger: ALTER TABLE ... REFRESH SCHEMA (schema registry update) where the new schema drops a column that a generated column expression references.

Common situations: Upstream producers remove fields from a protobuf/avro/json payload registered in the schema registry while the RW table has computed columns depending on those fields.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• failed to drop column "{}" because it's referenced by a gene
• request error
• payload shorter than 18-byte glue header
• Only support glue header version 3 but found {}
• unused FORMAT ENCODE option: key.message='{name}'

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/6565301e66299a7c. Report an issue: GitHub.