risingwavelabs/risingwave · error · SinkError::BigQuery

Cannot find column_name

Error message

Cannot find column_name

What it means

During BigQuery sink validation, RisingWave queries INFORMATION_SCHEMA.COLUMNS and reads each row via the gcp-bigquery-client ResultSet. This error is thrown when the 'column_name' column exists in the query result but its value for this row is NULL, so the map of the remote table schema cannot be built. It indicates an unexpected/NULL schema response from BigQuery for the target table.

Solutions

  1. Confirm the target table is healthy: run `SELECT column_name, data_type FROM <project>.<dataset>.INFORMATION_SCHEMA.COLUMNS WHERE table_name='<table>'` in BigQuery console and check for NULLs.
  2. If the table is corrupt or unexpected, drop and recreate it (or set auto_create=true so RisingWave recreates it).
  3. Verify the service account has metadata read permission on the dataset so the correct COLUMNS view is returned.
  4. Pin/upgrade the gcp-bigquery-client crate and retry validation.
Defensive patterns

Strategy: validation

Validate before calling

-- Run against BigQuery before creating the sink:
SELECT column_name, data_type
FROM `<project>.<dataset>.INFORMATION_SCHEMA.COLUMNS`
WHERE table_name = '<table>';  -- ensure no NULL column_name rows

Try / catch

match sink_creation {
    Err(SinkError::BigQuery(e)) if e.to_string().contains("Cannot find column_name") => {
        // inspect/repair remote table metadata, then retry
    }
    other => other?,
}

Prevention

When it happens

Trigger: Calling validate() on a BigQuery sink where the INFORMATION_SCHEMA.COLUMNS query returns a row whose column_name value is NULL - e.g. a corrupted or partially-created table entry, or a query result whose rows contain null column_name values.

Common situations: Target table exists but was created/altered externally in an inconsistent state; querying a shadow/system table with NULL column names; stale or mocked responses.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/c09fe0935825f275. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/sink/big_query.rs:614

        let rs = client
            .job()
            .query(
                &self.config.common.project,
                QueryRequest::new(format!(
                    "SELECT column_name, data_type FROM `{}.{}.INFORMATION_SCHEMA.COLUMNS` WHERE table_name = '{}'",
                    project_id, dataset_id, table_id,
                )),
            ).await.map_err(|e| SinkError::BigQuery(e.into()))?;
        let mut rs = ResultSet::new_from_query_response(rs);

        let mut big_query_schema = HashMap::default();
        while rs.next_row() {
            big_query_schema.insert(
                rs.get_string_by_name("column_name")
                    .map_err(|e| SinkError::BigQuery(e.into()))?
                    .ok_or_else(|| {
                        SinkError::BigQuery(anyhow::anyhow!("Cannot find column_name"))
                    })?,
                rs.get_string_by_name("data_type")
                    .map_err(|e| SinkError::BigQuery(e.into()))?
                    .ok_or_else(|| {
                        SinkError::BigQuery(anyhow::anyhow!("Cannot find column_name"))
                    })?,
            );
        }

        self.check_column_name_and_type(big_query_schema)?;
        Ok(())
    }
}

pub struct BigQuerySinkWriter {
    pub config: BigQueryConfig,
    #[expect(dead_code)]
    schema: Schema,

View on GitHub (pinned to 6469eb736d)