apache/hadoop · error · NoAwsCredentialsException

SimpleAWSCredentialsProvider: No AWS credentials in the Hado

Error message

SimpleAWSCredentialsProvider: No AWS credentials in the Hadoop configuration

What it means

NoAwsCredentialsException thrown by SimpleAWSCredentialsProvider.resolveCredentials() when the access key or secret key (fs.s3a.access.key / fs.s3a.secret.key, or their URI/password forms) is empty. This provider maps static long-term keys from Hadoop configuration; when either half is missing it reports it has nothing, and if no other provider in fs.s3a.aws.credentials.provider can authenticate, S3 calls fail.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SimpleAWSCredentialsProvider.java:85

  /**
   * Instantiate from a login tuple.
   * For testing, hence package-scoped.
   * @param login login secrets
   * @throws IOException failure
   */
  @VisibleForTesting
  SimpleAWSCredentialsProvider(final S3xLoginHelper.Login login)
      throws IOException {
    this.accessKey = login.getUser();
    this.secretKey = login.getPassword();
  }

  @Override
  public AwsCredentials resolveCredentials() {
    if (!StringUtils.isEmpty(accessKey) && !StringUtils.isEmpty(secretKey)) {
      return AwsBasicCredentials.create(accessKey, secretKey);
    }
    throw new NoAwsCredentialsException("SimpleAWSCredentialsProvider",
        "No AWS credentials in the Hadoop configuration");
  }

  @Override
  public String toString() {
    return "SimpleAWSCredentialsProvider{" +
        "accessKey.empty=" + accessKey.isEmpty() +
        ", secretKey.empty=" + secretKey.isEmpty() +
        '}';
  }

}

View on GitHub (pinned to 2add963021)

Solutions

1. Set both fs.s3a.access.key and fs.s3a.secret.key (config, URI, or Hadoop credential store)
2. If you intend environment/instance-profile auth, remove SimpleAWSCredentialsProvider from fs.s3a.aws.credentials.provider so the chain does not require static keys
3. Verify the credential store actually contains both entries: hadoop credential list -provider ...
4. Check for typos and per-bucket overrides that may blank the values

Example fix

<!-- before: provider in chain, keys missing -->
<property><name>fs.s3a.aws.credentials.provider</name>
  <value>org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider</value></property>

<!-- after: supply the pair ... -->
<property><name>fs.s3a.access.key</name><value>AKIA...</value></property>
<property><name>fs.s3a.secret.key</name><value>...</value></property>
<!-- ...or drop the provider entry and rely on IAM instance profiles -->

if (conf.get("fs.s3a.aws.credentials.provider", "")
    .contains("SimpleAWSCredentialsProvider")) {
  String ak = conf.get("fs.s3a.access.key", "");
  String sk = conf.get("fs.s3a.secret.key", "");
  if (ak.isEmpty() || sk.isEmpty()) {
    throw new IOException("SimpleAWSCredentialsProvider requires both"
        + " fs.s3a.access.key and fs.s3a.secret.key");
  }
}

catch NoAwsCredentialsException at the first S3 operation or fs init; treat as a configuration error - print the provider chain and where each property should come from; do not retry

When it happens

Trigger: fs.s3a.aws.credentials.provider includes SimpleAWSCredentialsProvider (or the default chain reaches it) but fs.s3a.access.key/fs.s3a.secret.key are unset, blank, or only one of the pair is set; secrets expected from a credential provider file that did not load.

Common situations: Removing static keys to move to IAM roles but leaving the simple provider in the chain; property name typos; JCEKS store missing on worker nodes; only one of the two keys migrated to the new cluster.

Related errors

• Cannot find password option {key}
• SSE-C is enabled but no encryption key was declared in fs.s3
• Session credentials in Hadoop configuration: No AWS Credenti
• Unset property fs.s3a.assumed.role.arn
• Class {className} AWS provider class cannot be used (configu

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ac7c5a1482618335. Report an issue: GitHub.