risingwavelabs/risingwave · error · SinkError::GooglePubSub

Failed to create Google Cloud Pub/Sub credentials file

Error message

Failed to create Google Cloud Pub/Sub credentials file

What it means

When `credentials` are configured, the sink parses the service-account JSON string with gcloud-sdk's CredentialsFile::new_from_str. If the string is not valid credentials JSON (bad key fields, truncated copy, wrong value), the underlying error is wrapped with the context 'Failed to create Google Cloud Pub/Sub credentials file'.

Source

Thrown at src/connector/src/sink/google_pubsub.rs:223

    add_future: DeliveryFutureManagerAddFuture<'w, GooglePubSubSinkDeliveryFuture>,
}

impl GooglePubSubSinkWriter {
    pub async fn new(
        config: GooglePubSubConfig,
        schema: Schema,
        pk_indices: Vec<usize>,
        format_desc: &SinkFormatDesc,
        db_name: String,
        sink_from_name: String,
    ) -> Result<Self> {
        let environment = if let Some(ref cred) = config.credentials {
            let mut auth_config = project::Config::default();
            auth_config = auth_config.with_audience(apiv1::conn_pool::AUDIENCE);
            auth_config = auth_config.with_scopes(&apiv1::conn_pool::SCOPES);
            let cred_file = CredentialsFile::new_from_str(cred).await.map_err(|e| {
                SinkError::GooglePubSub(
                    anyhow!(e).context("Failed to create Google Cloud Pub/Sub credentials file"),
                )
            })?;
            let provider =
                DefaultTokenSourceProvider::new_with_credentials(auth_config, Box::new(cred_file))
                    .await
                    .map_err(|e| {
                        SinkError::GooglePubSub(
                            anyhow!(e).context(
                                "Failed to create Google Cloud Pub/Sub token source provider",
                            ),
                        )
                    })?;
            Environment::GoogleCloud(Box::new(provider))
        } else if let Some(emu_host) = config.emulator_host {
            Environment::Emulator(emu_host)
        } else {
            return Err(SinkError::GooglePubSub(anyhow!(
                "Missing emulator_host or credentials in Google Pub/Sub sink"

View on GitHub (pinned to 6469eb736d)

Solutions

1. Inspect the inner error chained under this message to see which credential field failed validation
2. Provide the full, unmodified service-account key JSON as the credentials value
3. Validate the JSON locally (e.g. `jq . key.json` and check fields like client_email/private_key) and re-create the sink

Example fix

// before
pubsub.credentials='{"type": "service_account", "client_email": ...'  -- truncated
// after
pubsub.credentials='{"type":"service_account","project_id":"p","private_key_id":"..","private_key":"-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n","client_email":"sa@p.iam.gserviceaccount.com", ...}'

fn validate_credentials_json(cred: &str) -> Result<(), String> {
    let v: serde_json::Value = serde_json::from_str(cred).map_err(|e| format!("credentials is not valid JSON: {e}"))?;
    for field in ["type", "private_key", "client_email"] {
        if v.get(field).is_none() { return Err(format!("credentials missing field: {field}")); }
    }
    Ok(())
}

When it happens

Trigger: `pubsub.credentials` property contains malformed service-account JSON, e.g. copied with surrounding quotes/escapes, truncated, or a non-credential JSON object.

Common situations: Secrets managers injecting escaped/encoded JSON; pasting only part of a key file; using an API key instead of a service-account key JSON.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Related errors

• Failed to create Google Cloud Pub/Sub token source provider
• Missing emulator_host or credentials in Google Pub/Sub sink
• Google Pub/Sub error: {0}
• credentials must be set if not using the pubsub emulator
• subscription {} does not exist

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/fce69bf8fd2d25b7. Report an issue: GitHub.