databendlabs/databend · error · InvalidInput

s3.access-key-id and s3.secret-access-key must be…

Error message

s3.access-key-id and s3.secret-access-key must be configured together

What it means

IcebergFileIO::validate_s3_credentials enforces that S3 static credentials are complete before building an opendal Operator. The props map from the iceberg FileIO must contain s3.access-key-id and s3.secret-access-key as a pair; providing only one is rejected as InvalidInput because partial credentials can never authenticate.

Solutions

Add the missing counterpart property: set both s3.access-key-id and s3.secret-access-key in the FileIO/catalog properties.
If you intended no static credentials (e.g. IAM role or anonymous access), remove the half-configured key entirely instead of setting only one.
Verify the properties source (catalog config, environment substitution, secret mount) actually injects both values.

Example fix

// before (iceberg FileIO props)
props.insert("s3.access-key-id".into(), "AKIA...".into());
// after
props.insert("s3.access-key-id".into(), "AKIA...".into());
props.insert("s3.secret-access-key".into(), "secret...".into());

Defensive patterns

Strategy: validation

Validate before calling

let has_ak = props.contains_key("s3.access-key-id");
let has_sk = props.contains_key("s3.secret-access-key");
if has_ak != has_sk {
    return Err("s3.access-key-id and s3.secret-access-key must be set together".into());
}

Try / catch

// map the InvalidInput to a config error with context
match file_io.get_operator_path(location) {
    Err(e) if e.message().contains("configured together") => {
        return Err(ConfigError::IncompleteS3Credentials(location.to_string()));
    }
    other => other,
}

Prevention

Always set S3 credentials as a matched pair, sourced from the same secret/config block.
Prefer IAM roles / credential chain over static keys when possible.
Validate FileIO props at catalog configuration time, before table operations.

When it happens

Trigger: Building an IcebergFileIO operator (build_operator -> validate_s3_credentials) whose FileIO props contain s3.access-key-id but no s3.secret-access-key, or vice versa.

Common situations: Misconfigured S3 catalog/table properties where a user sets only the access key ID; credential values injected from environment variables or secrets where one of the two failed to resolve; copy-pasting a properties block and dropping one line.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/84f3fe485db33e3e. Report an issue: GitHub.

Appendix: source

Thrown at src/common/storage/src/operator.rs:747

pub struct IcebergFileIO {
    scheme: String,
    props: std::collections::HashMap<String, String>,
}

impl IcebergFileIO {
    pub fn new(file_io: iceberg::io::FileIO) -> Self {
        let (scheme, props, _extensions) = file_io.into_builder().into_parts();
        Self { scheme, props }
    }

    fn validate_s3_credentials(&self) -> Result<()> {
        let has_access_key = self.props.contains_key("s3.access-key-id");
        let has_secret_key = self.props.contains_key("s3.secret-access-key");
        let has_session_token = self.props.contains_key("s3.session-token");

        if has_access_key != has_secret_key {
            return Err(Error::new(
                ErrorKind::InvalidInput,
                "s3.access-key-id and s3.secret-access-key must be configured together",
            ));
        }

        if has_session_token && !(has_access_key && has_secret_key) {
            return Err(Error::new(
                ErrorKind::InvalidInput,
                "s3.session-token requires s3.access-key-id and s3.secret-access-key",
            ));
        }

        Ok(())
    }

    fn build_operator(&self, location: &str) -> Result<(Operator, usize)> {
        let url = url::Url::parse(location)
            .map_err(|e| Error::new(ErrorKind::InvalidInput, e.to_string()))?;

View on GitHub (pinned to 288d84d76e)