apache/beam · error · IOException

AWS credential provider type '%s' is not supported

Error message

AWS credential provider type '%s' is not supported

What it means

AwsModule's credentials provider deserializer (deserializeWithType) reads the 'type' name from JSON and constructs the matching AwsCredentialsProvider. If the JSON type string matches none of the supported provider types (default, basic, profile, session, web-identity-token, sts-assume-role, etc.), it throws this IOException. It means serialized AWS options contain a credential provider type this Beam version cannot deserialize.

Source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/options/AwsModule.java:208

                StsClient.builder()
                    .region(Region.AWS_GLOBAL)
                    .credentialsProvider(AnonymousCredentialsProvider.create())
                    .build())
            .build();
      } else if (typeName.equals(
          StsAssumeRoleForFederatedCredentialsProvider.class.getSimpleName())) {
        return StsAssumeRoleForFederatedCredentialsProvider.builder()
            .setAudience(getNotNull(json, AUDIENCE, typeName))
            .setAssumedRoleArn(getNotNull(json, ROLE_ARN, typeName))
            .setWebIdTokenProviderFQCN(getNotNull(json, WEBID_TOKEN_FQCN, typeName))
            .setSessionDurationSecs(
                Optional.ofNullable(json.get(SESSION_DURATION_SECONDS))
                    .map(JsonNode::asInt)
                    .orElse(
                        StsAssumeRoleForFederatedCredentialsProvider.DEFAULT_SESSION_DURATION_SECS))
            .build();
      } else {
        throw new IOException(
            String.format("AWS credential provider type '%s' is not supported", typeName));
      }
    }

    private String getNotNull(JsonNode json, String key, String typeName) {
      JsonNode node = json.get(key);
      checkNotNull(node, "AWS credentials provider type '%s' is missing '%s'", typeName, key);
      return node.textValue();
    }

    private boolean hasName(Class<? extends AwsCredentialsProvider> clazz, String typeName) {
      return clazz.getSimpleName().equals(typeName);
    }
  }

  private static class AWSCredentialsProviderSerializer
      extends JsonSerializer<AwsCredentialsProvider> {
    // These providers are singletons, so don't require any serialization, other than type.

View on GitHub (pinned to 12126d8942)

Solutions

  1. Align Beam versions — run the pipeline with the same (or newer) Beam version that serialized the credentials provider.
  2. Use a supported provider type when configuring AwsOptions (e.g. DEFAULT, BASIC, PROFILE, SESSION, WEB_IDENTITY_TOKEN, STS_ASSUME_ROLE).
  3. Re-serialize the pipeline/options with the correct provider type instead of hand-editing JSON.

Example fix

// before (JSON)
{"type": "myCustomProvider", ...}
// after
{"type": "default"}
Defensive patterns

Strategy: validation

Validate before calling

// verify provider type before deserialization
Set<String> supported = Set.of("default","basic","profile","session","web-identity-token","sts-assume-role","sts-assume-role-with-web-identity","process");
if (!supported.contains(jsonType)) throw new IllegalArgumentException("Unsupported type: " + jsonType);

Try / catch

try {
  provider = deserializeAwsCredentialsProvider(json);
} catch (IOException e) {
  // fall back to DefaultCredentialsProvider and warn
}

Prevention

When it happens

Trigger: Deserializing AwsCredentialsProvider JSON (e.g. from serialized pipeline options or job submission) whose type field was produced by a different Beam version or manually crafted with an unknown type value.

Common situations: Beam version mismatch between pipeline authoring and runner: newer Beam serialized a provider type the older AwsModule doesn't know; hand-edited pipeline option JSON; typo in the credential provider type name.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/97a992d646b800f4. Report an issue: GitHub.