risingwavelabs/risingwave · error · SinkError

s3.access.key and s3.secret.key is required with aws s3

Error message

s3.access.key and s3.secret.key is required with aws s3

What it means

Raised while building the AWS S3 (or S3-compatible) configuration for a DeltaLake sink. After resolving AWS credentials from the user's `aws_auth_props`, the SDK returned no credentials provider, meaning static access/secret keys were not configured. DeltaLake S3 writes require concrete static credentials; assumed-role or instance-profile resolution without them is rejected here.

Source

Thrown at src/connector/src/sink/deltalake.rs:151

            Ok(DeltaTableUrl::Gcs(path.to_owned()))
        } else if let Some(path) = path.strip_prefix("file://") {
            Ok(DeltaTableUrl::Local(path.to_owned()))
        } else {
            Err(SinkError::DeltaLake(anyhow!(
                "path should start with 's3://','s3a://'(s3) ,gs://(gcs) or file://(local)"
            )))
        }
    }

    async fn build_delta_lake_config_for_aws(&self) -> Result<HashMap<String, String>> {
        let mut storage_options = HashMap::new();
        storage_options.insert(AWS_ALLOW_HTTP.to_owned(), "true".to_owned());
        storage_options.insert(AWS_S3_ALLOW_UNSAFE_RENAME.to_owned(), "true".to_owned());
        let sdk_config = self.aws_auth_props.build_config().await?;
        let credentials = sdk_config
            .credentials_provider()
            .ok_or_else(|| {
                SinkError::Config(anyhow!(
                    "s3.access.key and s3.secret.key is required with aws s3"
                ))
            })?
            .as_ref()
            .provide_credentials()
            .await
            .map_err(|e| SinkError::Config(e.into()))?;
        let region = sdk_config.region();
        let endpoint = sdk_config.endpoint_url();
        storage_options.insert(
            AWS_ACCESS_KEY_ID.to_owned(),
            credentials.access_key_id().to_owned(),
        );
        storage_options.insert(
            AWS_SECRET_ACCESS_KEY.to_owned(),
            credentials.secret_access_key().to_owned(),
        );
        if endpoint.is_none() && region.is_none() {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Add `s3.access.key` and `s3.secret.key` to the sink WITH options
2. Verify property names are spelled exactly (no typos, correct dots)
3. If using temporary credentials, also supply `s3.session.token`
4. Alternatively configure an S3-compatible endpoint mode where credentials come from other supported sources

Example fix

// before
WITH (
  connector = 'deltalake',
  location = 's3://bucket/table',
  s3.region = 'us-east-1'
)
// after
WITH (
  connector = 'deltalake',
  location = 's3://bucket/table',
  s3.access.key = 'AKIA...',
  s3.secret.key = '...',
  s3.region = 'us-east-1'
)

fn validate_s3_creds(props: &BTreeMap<String, String>) -> Result<(), String> {
    if props.get("s3.access.key").map_or(true, |v| v.is_empty())
        || props.get("s3.secret.key").map_or(true, |v| v.is_empty()) {
        return Err("s3.access.key and s3.secret.key must be set and non-empty".into());
    }
    Ok(())
}

When it happens

Trigger: Creating a DeltaLake sink with `connector='s3'` (aws mode) where `s3.access.key` / `s3.secret.key` are absent from WITH properties, so `aws_auth_props.build_config()` yields a config whose `credentials_provider()` is None.

Common situations: Users relying on IAM roles or environment credentials instead of explicit keys; typos in the `s3.access.key`/`s3.secret.key` property names; copying a config template and leaving key fields blank.

Related errors

• s3.endpoint and s3.region need to be filled with at least on
• Both `access_key` and `secret_key` must be provided
• gcs.service.account is required with Google Cloud Storage (G
• s3 url {location} should have a '/' at the start of path.
• Url parse error for S3 deltalake location (wrapped error)

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/96462d2b17d9b90f. Report an issue: GitHub.