risingwavelabs/risingwave · error

`{}` column not found in source backfill state table schema

Error message

`{}` column not found in source backfill state table schema

What it means

When building the plan for `internal_source_backfill_progress()`, each source backfill state table must contain a column named BACKFILL_PROGRESS_COLUMN_NAME so the rule can project its index. If the column lookup by name fails, the table schema does not match the expected internal source-backfill table shape. This is an internal schema invariant of the auto-generated state table.

Source

Thrown at src/frontend/src/optimizer/rule/table_function_to_internal_source_backfill_progress.rs:160

struct SourceBackfillInfo {
    job_id: JobId,
    fragment_id: FragmentId,
    table_id: TableId,
    partition_id_column_index: usize,
    backfill_progress_column_index: usize,
}

impl SourceBackfillInfo {
    fn new(table: &TableCatalog) -> anyhow::Result<Self> {
        let Some(job_id) = table.job_id else {
            bail!("`job_id` column not found in source backfill table catalog");
        };
        let Some(backfill_progress_column_index) = table
            .columns
            .iter()
            .position(|c| c.name() == StreamSourceScan::BACKFILL_PROGRESS_COLUMN_NAME)
        else {
            bail!(
                "`{}` column not found in source backfill state table schema",
                StreamSourceScan::BACKFILL_PROGRESS_COLUMN_NAME
            );
        };
        let Some(partition_id_column_index) = table
            .columns
            .iter()
            .position(|c| c.name() == StreamSourceScan::PARTITION_ID_COLUMN_NAME)
        else {
            bail!(
                "`{}` column not found in source backfill state table schema",
                StreamSourceScan::PARTITION_ID_COLUMN_NAME
            );
        };
        let fragment_id = table.fragment_id;
        let table_id = table.id;

        Ok(Self {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Upgrade/complete the in-flight source backfill jobs so their state tables are recreated with the current schema; check `rw_databases`/internal table schema via `SHOW COLUMNS FROM` on the internal table id.
2. Drop and recreate the source/backfilling materialized view or table whose internal state table has the stale schema.
3. If a version-skew left orphaned tables, clean them up (drop leftover jobs) and re-run the progress query.
4. For developers: harden the filter in `get_source_backfilling_tables` to also verify the expected columns before building SourceBackfillInfo.

-- verify internal state table schema before querying progress
SHOW COLUMNS FROM <internal_backfill_table_id>;

// Rust: check schema shape up front
fn has_required_columns(t: &TableCatalog) -> bool {
    t.columns.iter().any(|c| c.name() == StreamSourceScan::BACKFILL_PROGRESS_COLUMN_NAME)
}

// catch and report which table failed
if let Err(e) = SourceBackfillInfo::new(&table) { return Err(e.context(format!("table {}: {}", table.id, e))) }

When it happens

Trigger: Running `SELECT * FROM internal_source_backfill_progress()` while a source-backfill state table in the catalog lacks the backfill progress column — e.g. the table name matched `is_source_backfill_table` but was created by an older/other code path without the progress column.

Common situations: Cluster upgraded from a RisingWave version whose source-backfill state tables had a different schema while old internal tables persist; manual edits or migrations of internal tables; a misclassified table passing the name-based `is_source_backfill_table` filter.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• `{}` column not found in backfill table
• `job_id` column not found in source backfill table catalog
• `job_id` column not found in backfill table
• progress table columns len not matched with the len derived
• progress table column not matched with upstream table schema

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/0af98c6c83a01d0e. Report an issue: GitHub.