apache/hadoop · error · PathIOException

Unset property fs.s3a.assumed.role.arn

Error message

Unset property fs.s3a.assumed.role.arn

What it means

PathIOException (E_NO_ROLE) thrown from the AssumedRoleCredentialProvider constructor: the provider is listed in fs.s3a.aws.credentials.provider but fs.s3a.assumed.role.arn is empty. Assumed-role authentication always needs a role ARN to assume, so construction fails immediately.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/AssumedRoleCredentialProvider.java:110

  private final StsClient stsClient;

  /**
   * Instantiate.
   * This calls {@link #resolveCredentials()} to fail fast on the inner
   * role credential retrieval.
   * @param fsUri possibly null URI of the filesystem.
   * @param conf configuration
   * @throws IOException on IO problems and some parameter checking
   * @throws IllegalArgumentException invalid parameters
   * @throws StsException problems getting credentials
   */
  public AssumedRoleCredentialProvider(@Nullable URI fsUri, Configuration conf)
      throws IOException {

    arn = conf.getTrimmed(ASSUMED_ROLE_ARN, "");
    if (StringUtils.isEmpty(arn)) {
      throw new PathIOException(String.valueOf(fsUri), E_NO_ROLE);
    }

    // build up the base provider
    credentialsToSTS = buildAWSProviderList(fsUri, conf,
        ASSUMED_ROLE_CREDENTIALS_PROVIDER,
        Arrays.asList(
            SimpleAWSCredentialsProvider.class,
            EnvironmentVariableCredentialsProvider.class),
        Sets.newHashSet(getClass()));
    LOG.debug("Credentials used to obtain role credentials: {}", credentialsToSTS);

    // then the STS binding
    sessionName = conf.getTrimmed(ASSUMED_ROLE_SESSION_NAME,
        buildSessionName());
    duration = conf.getTimeDuration(ASSUMED_ROLE_SESSION_DURATION,
        ASSUMED_ROLE_SESSION_DURATION_DEFAULT, TimeUnit.SECONDS);
    String policy = conf.getTrimmed(ASSUMED_ROLE_POLICY, "");
    String externalId = conf.getTrimmed(ASSUMED_ROLE_EXTERNAL_ID, "");

View on GitHub (pinned to 2add963021)

Solutions

1. Set fs.s3a.assumed.role.arn to the full role ARN, e.g. arn:aws:iam::123456789012:role/s3-reader
2. Check the exact property name and bucket-scoped variant (fs.s3a.bucket.<bucket>.assumed.role.arn)
3. Ensure the ARN is visible where the filesystem is initialized - job configs can override cluster configs
4. If assumed roles are not intended, remove AssumedRoleCredentialProvider from the chain

Example fix

<!-- before -->
<property><name>fs.s3a.aws.credentials.provider</name>
  <value>org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider</value></property>

<!-- after -->
<property><name>fs.s3a.aws.credentials.provider</name>
  <value>org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider</value></property>
<property><name>fs.s3a.assumed.role.arn</name>
  <value>arn:aws:iam::123456789012:role/s3-reader</value></property>

boolean usesRole = conf.get("fs.s3a.aws.credentials.provider", "")
    .contains("AssumedRoleCredentialProvider");
if (usesRole && conf.getTrimmed("fs.s3a.assumed.role.arn", "").isEmpty()) {
  throw new IOException("fs.s3a.assumed.role.arn must be set when"
      + " AssumedRoleCredentialProvider is in the provider chain");
}

catch PathIOException with 'Unset property fs.s3a.assumed.role.arn' at fs init; set the ARN or remove the provider - not retryable

When it happens

Trigger: Adding org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider to fs.s3a.aws.credentials.provider without setting fs.s3a.assumed.role.arn, or with a whitespace-only value, or when a bucket-scoped config enables the provider but misses the ARN.

Common situations: Copy-pasted provider lists from documentation; property name typos (fs.s3a.assume.role.arn); job-level config overriding the cluster config and dropping the ARN; templates that parameterize the provider but not the role.

Related errors

• Class {className} AWS provider class cannot be used (configu
• Cannot find password option {key}
• SSE-C is enabled but no encryption key was declared in fs.s3
• SimpleAWSCredentialsProvider: No AWS credentials in the Hado
• Provider {this} has no credentials: {initializationException

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/791ad61e71ad8b78. Report an issue: GitHub.