apache/hadoop · error · NoAwsCredentialsException

SimpleAWSCredentialsProvider: No AWS credentials in the…

Error message

SimpleAWSCredentialsProvider: No AWS credentials in the Hadoop configuration

What it means

NoAwsCredentialsException thrown by SimpleAWSCredentialsProvider.resolveCredentials() when the access key or secret key (fs.s3a.access.key / fs.s3a.secret.key, or their URI/password forms) is empty. This provider maps static long-term keys from Hadoop configuration; when either half is missing it reports it has nothing, and if no other provider in fs.s3a.aws.credentials.provider can authenticate, S3 calls fail.

Solutions

Set both fs.s3a.access.key and fs.s3a.secret.key (config, URI, or Hadoop credential store)
If you intend environment/instance-profile auth, remove SimpleAWSCredentialsProvider from fs.s3a.aws.credentials.provider so the chain does not require static keys
Verify the credential store actually contains both entries: hadoop credential list -provider ...
Check for typos and per-bucket overrides that may blank the values

Example fix

<!-- before: provider in chain, keys missing -->
<property><name>fs.s3a.aws.credentials.provider</name>
  <value>org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider</value></property>

<!-- after: supply the pair ... -->
<property><name>fs.s3a.access.key</name><value>AKIA...</value></property>
<property><name>fs.s3a.secret.key</name><value>...</value></property>
<!-- ...or drop the provider entry and rely on IAM instance profiles -->

Defensive patterns

Strategy: validation

Validate before calling

if (conf.get("fs.s3a.aws.credentials.provider", "")
    .contains("SimpleAWSCredentialsProvider")) {
  String ak = conf.get("fs.s3a.access.key", "");
  String sk = conf.get("fs.s3a.secret.key", "");
  if (ak.isEmpty() || sk.isEmpty()) {
    throw new IOException("SimpleAWSCredentialsProvider requires both"
        + " fs.s3a.access.key and fs.s3a.secret.key");
  }
}

Try / catch

catch NoAwsCredentialsException at the first S3 operation or fs init; treat as a configuration error - print the provider chain and where each property should come from; do not retry

Prevention

Keep the provider chain and the credential properties it needs in one reviewed config unit
Prefer Hadoop credential stores over XML for static keys
On EC2, prefer the default chain with the IAM instance profile provider

When it happens

Trigger: fs.s3a.aws.credentials.provider includes SimpleAWSCredentialsProvider (or the default chain reaches it) but fs.s3a.access.key/fs.s3a.secret.key are unset, blank, or only one of the pair is set; secrets expected from a credential provider file that did not load.

Common situations: Removing static keys to move to IAM roles but leaving the simple provider in the chain; property name typos; JCEKS store missing on worker nodes; only one of the two keys migrated to the new cluster.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ac7c5a1482618335. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SimpleAWSCredentialsProvider.java:85

  /**
   * Instantiate from a login tuple.
   * For testing, hence package-scoped.
   * @param login login secrets
   * @throws IOException failure
   */
  @VisibleForTesting
  SimpleAWSCredentialsProvider(final S3xLoginHelper.Login login)
      throws IOException {
    this.accessKey = login.getUser();
    this.secretKey = login.getPassword();
  }

  @Override
  public AwsCredentials resolveCredentials() {
    if (!StringUtils.isEmpty(accessKey) && !StringUtils.isEmpty(secretKey)) {
      return AwsBasicCredentials.create(accessKey, secretKey);
    }
    throw new NoAwsCredentialsException("SimpleAWSCredentialsProvider",
        "No AWS credentials in the Hadoop configuration");
  }

  @Override
  public String toString() {
    return "SimpleAWSCredentialsProvider{" +
        "accessKey.empty=" + accessKey.isEmpty() +
        ", secretKey.empty=" + secretKey.isEmpty() +
        '}';
  }

}

View on GitHub (pinned to 2add963021)