risingwavelabs/risingwave · error · SinkError::Config

Primary key not defined for upsert bigquery sink (please def

Error message

Primary key not defined for upsert bigquery sink (please define in `primary_key` field)

What it means

`validate` enforces that a non-append-only BigQuery sink has a primary key: if the sink is not append-only and `pk_indices` is empty, upserts cannot be keyed and the sink raises `SinkError::Config`. BigQuery upsert mode requires a primary key to identify rows to update.

Source

Thrown at src/connector/src/sink/big_query.rs:557

            self.config.clone(),
            self.schema.clone(),
            self.pk_indices.clone(),
            self.is_append_only,
        )
        .await?;
        Ok(BigQueryLogSinker::new(
            writer,
            resp_stream,
            BIGQUERY_SEND_FUTURE_BUFFER_MAX_SIZE,
        ))
    }

    async fn validate(&self) -> Result<()> {
        risingwave_common::license::Feature::BigQuerySink
            .check_available()
            .map_err(|e| anyhow::anyhow!(e))?;
        if !self.is_append_only && self.pk_indices.is_empty() {
            return Err(SinkError::Config(anyhow!(
                "Primary key not defined for upsert bigquery sink (please define in `primary_key` field)"
            )));
        }
        let client = self
            .config
            .common
            .build_client(&self.config.aws_auth_props)
            .await?;
        let BigQueryCommon {
            project: project_id,
            dataset: dataset_id,
            table: table_id,
            ..
        } = &self.config.common;

        if self.config.common.auto_create {
            match client
                .table()

View on GitHub (pinned to 6469eb736d)

Solutions

Add `primary_key = '<col,...>'` to the sink's WITH/options so upserts can be keyed.
If the data is actually append-only, create the sink in append-only mode so no PK is required.
Ensure the underlying table/MV has a primary key and reference those columns as the sink's primary key.

Example fix

-- before
CREATE SINK s FROM mv WITH (
  connector='bigquery', ..., type='upsert'
);

-- after
CREATE SINK s FROM mv WITH (
  connector='bigquery', ..., type='upsert', primary_key='id'
);

Defensive patterns

Strategy: validation

Validate before calling

-- Ensure the source MV/table has a PK and you specify primary_key for upsert sinks:
SELECT relation, primary_key
FROM rw_catalog.rw_relations
WHERE name = '<mv_name>';
-- Only use type='upsert' when primary_key is set in WITH options.

Try / catch

// Catch config error on sink creation
try {
  await client.query("CREATE SINK s FROM mv WITH (connector='bigquery', type='upsert', primary_key='id')");
} catch (e) {
  if (e.message.includes('Primary key not defined for upsert')) {
    // retry with primary_key option or switch to append-only type
  }
  throw e;
}

Prevention

Always pair `type='upsert'` (or non-append-only sinks) with an explicit `primary_key` option.
Confirm the source table/MV has a primary key before configuring upsert sinks.
Use append-only sink mode when you only need inserts and no dedup/upserts.

When it happens

Trigger: Creating a BigQuery sink with upsert semantics (non-append-only, e.g. from a table or MV with PKs not propagated, or explicit non-append-only mode) without specifying `primary_key` in the sink's WITH options.

Common situations: Users sink non-append-only materialized views to BigQuery for upserts but omit the `primary_key` option; MVs without PKs are sunk in upsert mode by mistake.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/e9327201d3344a48. Report an issue: GitHub.