risingwavelabs/risingwave · error · SinkError::GooglePubSub

Configure at least one of `pubsub.emulator_host` and `pubsub

Error message

Configure at least one of `pubsub.emulator_host` and `pubsub.credentials` in the Google Pub/Sub sink

What it means

The Pub/Sub sink requires authentication configuration: either an emulator host (for local testing) or service-account credentials (for real GCP). validate() checks that at least one of `emulator_host` or `credentials` is present and fails the sink creation otherwise, so misconfigured sinks are caught before runtime.

Source

Thrown at src/connector/src/sink/google_pubsub.rs:158

    }
}
impl Sink for GooglePubSubSink {
    type LogSinker = AsyncTruncateLogSinkerOf<GooglePubSubSinkWriter>;

    const SINK_NAME: &'static str = PUBSUB_SINK;

    crate::impl_validate_sink_unknown_fields!();

    async fn validate(&self) -> Result<()> {
        if !self.is_append_only {
            return Err(SinkError::GooglePubSub(anyhow!(
                "Google Pub/Sub sink only support append-only mode"
            )));
        }

        let conf = &self.config;
        if matches!((&conf.emulator_host, &conf.credentials), (None, None)) {
            return Err(SinkError::GooglePubSub(anyhow!(
                "Configure at least one of `pubsub.emulator_host` and `pubsub.credentials` in the Google Pub/Sub sink"
            )));
        }

        Ok(())
    }

    async fn new_log_sinker(&self, _writer_param: SinkWriterParam) -> Result<Self::LogSinker> {
        Ok(GooglePubSubSinkWriter::new(
            self.config.clone(),
            self.schema.clone(),
            self.pk_indices.clone(),
            &self.format_desc,
            self.db_name.clone(),
            self.sink_from_name.clone(),
        )
        .await?
        .into_log_sinker(PUBSUB_SEND_FUTURE_BUFFER_MAX_SIZE))

View on GitHub (pinned to 6469eb736d)

Solutions

Add `pubsub.credentials='<service-account-json>'` for production GCP usage
Set `pubsub.emulator_host='localhost:8085'` when testing against the Pub/Sub emulator
Verify exact option key names against GooglePubSubConfig to avoid typos that leave both fields None

Example fix

// before
WITH (connector='google_pubsub', topic='mytopic')
// after
WITH (connector='google_pubsub', topic='mytopic', pubsub.credentials='<sa-json>')

Defensive patterns

Strategy: validation

Validate before calling

fn validate_pubsub_auth(props: &std::collections::BTreeMap<String, String>) -> Result<(), String> {
    let has_emu = props.contains_key("pubsub.emulator_host");
    let has_cred = props.contains_key("pubsub.credentials");
    if !has_emu && !has_cred {
        return Err("Set pubsub.emulator_host or pubsub.credentials".into());
    }
    Ok(())
}

Prevention

Store credentials in a secret manager and template them into sink DDL
Use emulator_host in dev and credentials in prod, checking per environment
Search sink DDL for the exact key names pubsub.emulator_host / pubsub.credentials before applying

When it happens

Trigger: CREATE SINK with connector='google_pubsub' whose properties include neither `pubsub.emulator_host` nor `pubsub.credentials`.

Common situations: First-time users forgetting GCP credentials; removing the credentials line when moving configs between environments; typos in the property key so neither field is populated.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/9417f65981542d1d. Report an issue: GitHub.