risingwavelabs/risingwave · error · SinkError::BigQuery

Cannot find table in bigquery

Error message

Cannot find table in bigquery

What it means

During sink validation, `check_column_name_and_type` fetches the existing BigQuery table's column descriptions. If the returned map is empty, BigQuery returned no table schema — typically because the table does not exist in the dataset — and the sink aborts with "Cannot find table in bigquery".

Source

Thrown at src/connector/src/sink/big_query.rs:359

            "NUMERIC" | "BIGNUMERIC"
        )
    }

    fn is_data_type_compatible(rw_data_type: &DataType, bigquery_type: &str) -> Result<bool> {
        if matches!(rw_data_type, DataType::Decimal) {
            return Ok(Self::is_decimal_type_compatible(bigquery_type));
        }

        Ok(Self::get_string_and_check_support_from_datatype(rw_data_type)? == bigquery_type)
    }

    fn check_column_name_and_type(
        &self,
        big_query_columns_desc: HashMap<String, String>,
    ) -> Result<()> {
        let rw_fields_name = self.schema.fields();
        if big_query_columns_desc.is_empty() {
            return Err(SinkError::BigQuery(anyhow::anyhow!(
                "Cannot find table in bigquery"
            )));
        }
        if rw_fields_name.len().ne(&big_query_columns_desc.len()) {
            return Err(SinkError::BigQuery(anyhow::anyhow!(
                "The length of the RisingWave column {} must be equal to the length of the bigquery column {}",
                rw_fields_name.len(),
                big_query_columns_desc.len()
            )));
        }

        for i in rw_fields_name {
            let value = big_query_columns_desc.get(&i.name).ok_or_else(|| {
                SinkError::BigQuery(anyhow::anyhow!(
                    "Column `{:?}` on RisingWave side is not found on BigQuery side.",
                    i.name
                ))
            })?;

View on GitHub (pinned to 6469eb736d)

Solutions

1. Create the table in BigQuery before creating the sink (or let RW create it if the connector supports that with correct permissions)
2. Verify `bigquery.table`, `bigquery.dataset`, and the project in the service-account key match the intended GCP resource
3. Grant the service account BigQuery Data Editor / Metadata Viewer roles so it can read the table schema

Example fix

-- before: sink created before table exists
CREATE SINK s INTO bigquery.table = 'events' ...;
-- after: create table first, then sink
-- (run in GCP: bq mk --table mydataset.events ...)
CREATE SINK s INTO bigquery.table = 'events' ...;

bq show --format=json myproject:mydataset.events || echo "table missing; create it before CREATE SINK"

// treat empty schema map as a table-not-found precondition failure
if big_query_columns_desc.is_empty() {
    return Err(anyhow!("target BigQuery table not found; create it or fix bigquery.table/bigquery.dataset"));
}

When it happens

Trigger: Calling `validate` (sink creation) when the target `bigquery.table` does not exist in `bigquery.dataset`, the project/dataset name is wrong, or the service account lacks permission so the table lookup returns nothing.

Common situations: Typo in table or dataset name; creating the sink before creating the table in GCP; using a service account scoped to a different project; case-sensitive table-name mismatches.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• The length of the RisingWave column {} must be equal to the
• Column `{:?}` on RisingWave side is not found on BigQuery si
• schema invalid, record type required at top level of the sch
• PostgreSQL schema `{schema}` does not exist
• Invalid option: {message}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/b7d31c1143306f0a. Report an issue: GitHub.