risingwavelabs/risingwave · error · SinkError::BigQuery

Data type mismatch for column `{:?}`. BigQuery side: `{:?}`,

Error message

Data type mismatch for column `{:?}`. BigQuery side: `{:?}`, RisingWave side: `{:?}`.

What it means

For each matched column, `is_data_type_compatible` compares the BigQuery column's actual type against the type string that the connector would write for the RisingWave field (e.g. NUMERIC vs FLOAT64). A mismatch means the sink could write values BigQuery would reject or corrupt, so validation fails naming both types.

Source

Thrown at src/connector/src/sink/big_query.rs:380

        }
        if rw_fields_name.len().ne(&big_query_columns_desc.len()) {
            return Err(SinkError::BigQuery(anyhow::anyhow!(
                "The length of the RisingWave column {} must be equal to the length of the bigquery column {}",
                rw_fields_name.len(),
                big_query_columns_desc.len()
            )));
        }

        for i in rw_fields_name {
            let value = big_query_columns_desc.get(&i.name).ok_or_else(|| {
                SinkError::BigQuery(anyhow::anyhow!(
                    "Column `{:?}` on RisingWave side is not found on BigQuery side.",
                    i.name
                ))
            })?;
            let data_type_string = Self::get_string_and_check_support_from_datatype(&i.data_type)?;
            if !Self::is_data_type_compatible(&i.data_type, value)? {
                return Err(SinkError::BigQuery(anyhow::anyhow!(
                    "Data type mismatch for column `{:?}`. BigQuery side: `{:?}`, RisingWave side: `{:?}`. ",
                    i.name,
                    value,
                    data_type_string
                )));
            };
        }
        Ok(())
    }

    fn get_string_and_check_support_from_datatype(rw_data_type: &DataType) -> Result<String> {
        match rw_data_type {
            DataType::Boolean => Ok("BOOL".to_owned()),
            DataType::Int16 => Ok("INT64".to_owned()),
            DataType::Int32 => Ok("INT64".to_owned()),
            DataType::Int64 => Ok("INT64".to_owned()),
            DataType::Float32 => Err(SinkError::BigQuery(anyhow::anyhow!(
                "REAL is not supported for BigQuery sink. Please convert to FLOAT64 or other supported types."

View on GitHub (pinned to 6469eb736d)

Solutions

Alter/recreate the BigQuery table so each column's type equals the mapping produced by `get_string_and_check_support_from_datatype` (INT64, FLOAT64, NUMERIC, DATE, STRING, BYTES, JSON, ARRAY<...>)
Or cast the column in the sink's MV to a type compatible with the table
Recreate the sink to re-run validation

Example fix

-- BQ table column is FLOAT64 but RW side is DECIMAL
-- after: recreate BQ table column as NUMERIC, or
CREATE MATERIALIZED VIEW mv AS SELECT CAST(x AS DOUBLE PRECISION) AS x ...;

Defensive patterns

Strategy: validation

Validate before calling

const typeMap = { Int16: "INT64", Int32: "INT64", Int64: "INT64", Float64: "FLOAT64", Decimal: "NUMERIC", Date: "DATE", Varchar: "STRING", Bytea: "BYTES", Jsonb: "JSON" };
for (const f of rwFields) {
  const bqType = bqCols.get(f.name);
  if (typeMap[f.dataType] && bqType !== typeMap[f.dataType]) throw new Error(`type mismatch on ${f.name}: bq=${bqType} rw=${typeMap[f.dataType]}`);
}

Try / catch

if !is_compatible(rw_type, bq_type) {
    return Err(anyhow!("recreate table column {} as {}", name, expected_string));
}

Prevention

Let RisingWave create the table so types are chosen consistently
Compare bq show schema output against the RW mapping table before validating
Cast mismatched columns in the MV

When it happens

Trigger: Sink creation where a column exists on both sides but with incompatible types — e.g. RW INTEGER column against a BigQuery FLOAT64 column, RW VARCHAR against BYTES, or RW DECIMAL against a parameterized NUMERIC the compat check rejects.

Common situations: Table created manually with different types than RW would choose; NUMERIC/BIGNUMERIC parameterization differences; schema drift after MV evolution.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/a4055655a0a26b04. Report an issue: GitHub.