risingwavelabs/risingwave · error · SinkError::GooglePubSub

Failed to create Google Cloud Pub/Sub token source provider

Error message

Failed to create Google Cloud Pub/Sub token source provider

What it means

After loading the credentials file, the sink builds a DefaultTokenSourceProvider used to mint OAuth tokens for the Pub/Sub API. If token source construction fails (e.g. invalid key material, unusable service account), the error is wrapped with the context 'Failed to create Google Cloud Pub/Sub token source provider'.

Source

Thrown at src/connector/src/sink/google_pubsub.rs:231

        format_desc: &SinkFormatDesc,
        db_name: String,
        sink_from_name: String,
    ) -> Result<Self> {
        let environment = if let Some(ref cred) = config.credentials {
            let mut auth_config = project::Config::default();
            auth_config = auth_config.with_audience(apiv1::conn_pool::AUDIENCE);
            auth_config = auth_config.with_scopes(&apiv1::conn_pool::SCOPES);
            let cred_file = CredentialsFile::new_from_str(cred).await.map_err(|e| {
                SinkError::GooglePubSub(
                    anyhow!(e).context("Failed to create Google Cloud Pub/Sub credentials file"),
                )
            })?;
            let provider =
                DefaultTokenSourceProvider::new_with_credentials(auth_config, Box::new(cred_file))
                    .await
                    .map_err(|e| {
                        SinkError::GooglePubSub(
                            anyhow!(e).context(
                                "Failed to create Google Cloud Pub/Sub token source provider",
                            ),
                        )
                    })?;
            Environment::GoogleCloud(Box::new(provider))
        } else if let Some(emu_host) = config.emulator_host {
            Environment::Emulator(emu_host)
        } else {
            return Err(SinkError::GooglePubSub(anyhow!(
                "Missing emulator_host or credentials in Google Pub/Sub sink"
            )));
        };

        let client_config = ClientConfig {
            endpoint: config.endpoint,
            project_id: Some(config.project_id),
            environment,
            ..Default::default()

View on GitHub (pinned to 6469eb736d)

Solutions

1. Check the chained inner error for the root OAuth/key failure
2. Re-create the service-account key and update the `pubsub.credentials` value with fresh JSON
3. Verify the service account is enabled and has Pub/Sub Publisher permission; use the emulator to isolate auth issues

Example fix

// before
pubsub.credentials='<old revoked key json>'
// after
# create a new key in GCP Console -> IAM -> Service Accounts -> Keys, then:
pubsub.credentials='<fresh key json>'

// Pre-check: key must parse and service account should be active (do this in a dry-run with google-auth)
fn precheck_key_fields(cred: &str) -> Result<(), String> {
    let v: serde_json::Value = serde_json::from_str(cred).map_err(|e| e.to_string())?;
    if v["private_key"].as_str().map_or(true, |k| !k.contains("BEGIN PRIVATE KEY")) {
        return Err("private_key missing PEM markers (check newline escaping)".into());
    }
    Ok(())
}

match sink.new().await {
    Err(e) if e.to_string().contains("token source provider") => {
        // inspect chained source; refresh service-account key
        eprintln!("token source init failed: {e:#}");
    }
    other => other?,
}

When it happens

Trigger: Credentials JSON parses but the service account is invalid: revoked key, malformed private_key, wrong auth audience/scopes, or GCP-side rejection while initializing the token source.

Common situations: Rotated/deleted service-account keys still referenced in config; private_key with unescaped newlines; disabled service accounts in the GCP project.

Related errors

• Failed to create Google Cloud Pub/Sub credentials file
• Missing emulator_host or credentials in Google Pub/Sub sink
• Google Pub/Sub error: {0}
• credentials must be set if not using the pubsub emulator
• subscription {} does not exist

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/9828825820b0904a. Report an issue: GitHub.