apache/hadoop · error · NoAwsCredentialsException

Session credentials in Hadoop configuration: No AWS Credenti

Error message

Session credentials in Hadoop configuration: No AWS Credentials

What it means

NoAwsCredentialsException thrown by TemporaryAWSCredentialsProvider.createCredentials: the marshalled credentials read from configuration are not valid SessionOnly credentials. The provider reads fs.s3a.access.key/fs.s3a.secret.key/fs.s3a.session.key and deliberately treats having only long-term (non-session) credentials as empty, so a missing session token is the usual cause.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/TemporaryAWSCredentialsProvider.java:100

  /**
   * The credentials here must include a session token, else this operation
   * will raise an exception.
   * @param config the configuration
   * @return temporary credentials.
   * @throws IOException on any failure to load the credentials.
   * @throws NoAuthWithAWSException validation failure
   * @throws NoAwsCredentialsException the credentials are actually empty.
   */
  @Override
  protected AwsCredentials createCredentials(Configuration config)
      throws IOException {
    MarshalledCredentials creds = MarshalledCredentialBinding.fromFileSystem(
        getUri(), config);
    MarshalledCredentials.CredentialTypeRequired sessionOnly
        = MarshalledCredentials.CredentialTypeRequired.SessionOnly;
    // treat only having non-session creds as empty.
    if (!creds.isValid(sessionOnly)) {
      throw new NoAwsCredentialsException(COMPONENT);
    }
    return MarshalledCredentialBinding.toAWSCredentials(creds,
        sessionOnly, COMPONENT);
  }

}

View on GitHub (pinned to 2add963021)

Solutions

1. Provide all three properties: fs.s3a.access.key, fs.s3a.secret.key, and fs.s3a.session.key (the STS session token)
2. If you do not use session tokens, remove TemporaryAWSCredentialsProvider from fs.s3a.aws.credentials.provider
3. If tokens expire, refresh them into the credential store or config on a schedule
4. Check per-bucket overrides that may define the provider without the session key

Example fix

<!-- before: session provider in the chain, token missing -->
<property><name>fs.s3a.aws.credentials.provider</name>
  <value>org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider</value></property>

<!-- after: the full session triple -->
<property><name>fs.s3a.access.key</name><value>ASIA...</value></property>
<property><name>fs.s3a.secret.key</name><value>...</value></property>
<property><name>fs.s3a.session.key</name><value>FQoG...</value></property>

if (conf.get("fs.s3a.aws.credentials.provider", "")
    .contains("TemporaryAWSCredentialsProvider")) {
  String ak = conf.get("fs.s3a.access.key", "");
  String sk = conf.get("fs.s3a.secret.key", "");
  String st = conf.get("fs.s3a.session.key", "");
  if (ak.isEmpty() || sk.isEmpty() || st.isEmpty()) {
    throw new IOException("TemporaryAWSCredentialsProvider needs access key,"
        + " secret key AND session key");
  }
}

catch NoAwsCredentialsException from TemporaryAWSCredentialsProvider; it means session-credentials are incomplete - fix or drop the provider from the chain; not retryable unless tokens were mid-refresh

When it happens

Trigger: fs.s3a.aws.credentials.provider includes TemporaryAWSCredentialsProvider but fs.s3a.session.key is unset/blank; or only long-term access/secret keys are present, which fails the SessionOnly validation.

Common situations: Default provider chain left in place after switching to static keys; STS session credentials pasted incompletely (access+secret but no token); token stored in a credential store entry missing on some nodes; expired session where only the token was refreshed.

Related errors

• SimpleAWSCredentialsProvider: No AWS credentials in the Hado
• getCredentials failed: {e}
• {component}: No AWS credentials
• Cannot find password option {key}
• SSE-C is enabled but no encryption key was declared in fs.s3

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/41192026aa586d26. Report an issue: GitHub.