risingwavelabs/risingwave · error

credentials must be set if not using the pubsub emulator

Error message

credentials must be set if not using the pubsub emulator

What it means

The Google PubSub source enumerator client requires credentials to authenticate to the PubSub service. If neither `credentials` nor `emulator_host` is provided in the source properties, client construction bails with this error, because google-cloud-pubsub cannot build an authenticated client without them.

Source

Thrown at src/connector/src/source/google_pubsub/enumerator/client.rs:50

impl SplitEnumerator for PubsubSplitEnumerator {
    type Properties = PubsubProperties;
    type Split = PubsubSplit;

    async fn new(
        properties: Self::Properties,
        _context: SourceEnumeratorContextRef,
    ) -> ConnectorResult<PubsubSplitEnumerator> {
        if properties.parallelism.is_some() {
            tracing::warn!(
                "pubsub.parallelism is deprecated and will be ignored. \
                 Split count now adapts automatically to the number of actors."
            );
        }

        properties.subscriber_config()?;

        if properties.credentials.is_none() && properties.emulator_host.is_none() {
            bail!("credentials must be set if not using the pubsub emulator")
        }

        let sub = properties.subscription_client().await?;
        if !sub
            .exists(None)
            .await
            .context("error checking subscription validity")?
        {
            bail!("subscription {} does not exist", &sub.id())
        }

        let seek_to = match (properties.start_offset, properties.start_snapshot) {
            (None, None) => None,
            (Some(start_offset), None) => {
                let ts = start_offset
                    .parse::<i64>()
                    .context("error parsing start_offset")
                    .map(|nanos| Utc.timestamp_nanos(nanos).into())?;

View on GitHub (pinned to 6469eb736d)

Solutions

1. Add the service-account credentials JSON to the WITH clause: `credentials = '...json content or path...'` for the source.
2. For local development, set `pubsub.emulator_host = 'host:port'` in the WITH clause to skip real authentication.
3. Alternatively point to a credentials file path or set GOOGLE_APPLICATION_CREDENTIALS if the properties support ADC-based lookup.
4. Verify the credentials JSON is a valid service-account key (has private_key, client_email) since invalid JSON will fail later.

Example fix

-- before
CREATE SOURCE ps (...) WITH (
  connector = 'google_pubsub',
  pubsub.subscription = 'my-sub'
)
-- after
CREATE SOURCE ps (...) WITH (
  connector = 'google_pubsub',
  pubsub.subscription = 'my-sub',
  pubsub.credentials = '{"type": "service_account", ...}'
)

-- pre-check: exactly one of credentials/emulator_host present
SELECT
  (properties::jsonb ? 'pubsub.credentials') OR (properties::jsonb ? 'pubsub.emulator_host') AS auth_ok
FROM (SELECT '{...}' AS properties) t;

function hasPubsubAuth(props) {
  return Boolean(props.credentials) || Boolean(props.emulator_host);
}

match PubsubEnumeratorClient::new(props).await {
    Ok(c) => c,
    Err(e) if e.to_string().contains("credentials must be set") => {
        bail!("provide pubsub.credentials or pubsub.emulator_host: {e}")
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Creating a PubSub source with neither `credentials` (service-account JSON) nor `pubsub.emulator_host` set in the WITH clause, then calling PubsubEnumeratorClient::new.

Common situations: Local testing against the real service without supplying the service account key; deploying to an environment where ADC (Application Default Credentials) isn't set up and RisingWave requires explicit credentials; forgetting the credentials field after copying a config that used an emulator.

Related errors

• Google Pub/Sub error: {0}
• subscription {} does not exist
• specify at most one of start_offset or start_snapshot
• pubsub.ack_deadline_seconds must be between 10 and 600
• pubsub.max_outstanding_messages must be greater than 0

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/5ee193bfd669cfea. Report an issue: GitHub.