risingwavelabs/risingwave · error

missing aws credentials_provider

Error message

missing aws credentials_provider

What it means

When creating a Kafka source/sink with AWS MSK IAM authentication enabled, RisingWave builds an AWS SDK config and expects it to contain a credentials provider. If `build_config()` produced a config with no resolvable credentials, this error aborts context creation because IAM SASL/OAUTHBEARER signing is impossible without credentials.

Source

Thrown at src/connector/src/source/kafka/client_context.rs:67

    /// Credential and region for AWS MSK
    auth: Option<IamAuthEnv>,
}

impl KafkaContextCommon {
    pub async fn new(
        broker_rewrite_map: Option<BTreeMap<String, String>>,
        identifier: Option<String>,
        metrics: Option<Arc<RdKafkaStats>>,
        auth: AwsAuthProps,
        is_aws_msk_iam: bool,
    ) -> ConnectorResult<Self> {
        let addr_rewriter =
            BrokerAddrRewriter::new(PrivateLinkContextRole::Consumer, broker_rewrite_map)?;
        let auth = if is_aws_msk_iam {
            let config = auth.build_config().await?;
            let credentials_provider = config
                .credentials_provider()
                .ok_or_else(|| anyhow!("missing aws credentials_provider"))?;
            let region = config
                .region()
                .ok_or_else(|| anyhow!("missing aws region"))?
                .clone();
            Some(IamAuthEnv {
                credentials_provider,
                region,
                signer_timeout_sec: auth
                    .msk_signer_timeout_sec
                    .unwrap_or(Self::default_msk_signer_timeout_sec()),
            })
        } else {
            None
        };
        Ok(Self {
            addr_rewriter,
            identifier,
            metrics,

View on GitHub (pinned to 6469eb736d)

Solutions

1. Provide explicit credentials in the WITH options: set `aws.credentials.access_key_id`, `aws.credentials.secret_access_key` (and `aws.credentials.session_token` for temporary creds) alongside `aws.auth.msk_iam = true`
2. If relying on environment/instance credentials, verify the node runs where the default AWS credential chain resolves (env vars, ~/.aws/credentials, IMDS/IRSA) and the region is set (`aws.region` option)
3. Ensure the configured AWS profile exists and is readable; try the same credentials with `aws sts get-caller-identity` to confirm they resolve

Example fix

// before
CREATE SOURCE s WITH (
  connector='kafka',
  aws.auth.msk_iam='true'
) ...
// after
CREATE SOURCE s WITH (
  connector='kafka',
  aws.auth.msk_iam='true',
  aws.region='us-east-1',
  aws.credentials.access_key_id='AKIA...',
  aws.credentials.secret_access_key='...'
) ...

if msk_iam && (access_key_id.is_none() || secret_access_key.is_none()) && !ambient_aws_creds_available() {
    return Err("msk_iam requires explicit aws.credentials.access_key_id/secret_access_key or a resolvable default credential chain");
}

When it happens

Trigger: Creating a KafkaContextCommon with `is_aws_msk_iam=true` while the AwsAuthProps carry no usable credentials (no access_key/secret_key, no session_token where required, no profile, and no instance/IRSA role available).

Common situations: User sets `aws.auth.msk_iam = true` but forgets access/secret keys; running outside AWS (e.g. on-prem) where the default credential chain finds nothing; typo in the AWS profile name; IAM auth configured for an MSK cluster without any credentials source.

Related errors

• failed to generate AWS MSK IAM token
• missing aws region
• PrivateLink endpoint not found
• Both `access_key` and `secret_key` must be provided
• The number of broker addrs {} does not match the number of p

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/60f4f81b2078d221. Report an issue: GitHub.