apache/beam · error · IllegalArgumentException

Unsupported AWS credentials provider type

Error message

Unsupported AWS credentials provider type 

What it means

AwsModule's credentials provider serializer (serializeWithType) writes the provider to JSON; for provider classes that are neither composite (profile/session/etc.) nor in SINGLETON_CREDENTIAL_PROVIDERS, it throws this IllegalArgumentException. It means a custom AwsCredentialsProvider implementation was supplied that Beam's serializer does not know how to represent.

Source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/options/AwsModule.java:297

        Supplier<AssumeRoleWithWebIdentityRequest> reqSupplier =
            (Supplier<AssumeRoleWithWebIdentityRequest>)
                readField(credentialsProvider, "assumeRoleWithWebIdentityRequest");
        serializer
            .findValueSerializer(AssumeRoleWithWebIdentityRequest.serializableBuilderClass())
            .unwrappingSerializer(NameTransformer.NOP)
            .serialize(reqSupplier.get().toBuilder(), jsonGenerator, serializer);
      } else if (credentialsProvider instanceof StsAssumeRoleForFederatedCredentialsProvider) {
        StsAssumeRoleForFederatedCredentialsProvider provider =
            (StsAssumeRoleForFederatedCredentialsProvider) credentialsProvider;
        jsonGenerator.writeStringField(AUDIENCE, provider.audience());
        jsonGenerator.writeStringField(ROLE_ARN, provider.assumedRoleArn());
        jsonGenerator.writeStringField(WEBID_TOKEN_FQCN, provider.webIdTokenProviderFQCN());
        Integer sessionDurationSecs = provider.sessionDurationSecs();
        if (sessionDurationSecs != null) {
          jsonGenerator.writeNumberField(SESSION_DURATION_SECONDS, sessionDurationSecs);
        }
      } else if (!SINGLETON_CREDENTIAL_PROVIDERS.contains(providerClass)) {
        throw new IllegalArgumentException(
            "Unsupported AWS credentials provider type " + providerClass);
      }
      // BEAM-11958 Use deprecated Jackson APIs to be compatible with older versions of jackson
      typeSerializer.writeTypeSuffixForObject(credentialsProvider, jsonGenerator);
    }

    private Object readField(AwsCredentialsProvider provider, String fieldName) throws IOException {
      try {
        return FieldUtils.readField(provider, fieldName, true);
      } catch (IllegalArgumentException | IllegalAccessException e) {
        throw new IOException(
            String.format(
                "Failed to access private field '%s' of AWS credential provider type '%s' with reflection",
                fieldName, provider.getClass().getSimpleName()),
            e);
      }
    }
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Use one of Beam's supported provider types (DefaultCredentialsProvider, BasicAWSCredentialsProvider, ProfileCredentialsProvider, etc.).
2. If a custom provider is required, add it to SINGLETON_CREDENTIAL_PROVIDERS handling in AwsModule (fork/patch) or upstream it to Beam.
3. Configure credentials via static configuration (profile/keys/environment) instead of a custom provider object.

Example fix

// before
options.setAwsCredentialsProvider(myCustomProvider);
// after
options.setAwsCredentialsProvider(ProfileCredentialsProvider.create("my-profile"));

Class<?> c = provider.getClass();
if (!(c == DefaultCredentialsProvider.class || c == ProfileCredentialsProvider.class
    || c == EnvironmentVariableCredentialsProvider.class || c == SystemPropertyCredentialsProvider.class
    || c == ContainerCredentialsProvider.class || c == InstanceProfileCredentialsProvider.class)) {
  throw new IllegalArgumentException("Provider not serializable by Beam: " + c);
}

When it happens

Trigger: Setting AwsOptions.setAwsCredentialsProvider to a custom AwsCredentialsProvider implementation (or one from a different SDK extension) that is not one of the supported singleton types, then serializing the pipeline options.

Common situations: Users plugging a bespoke credentials provider into Beam's AwsOptions and submitting the job — serialization of the pipeline fails at job submission time.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• can not be serialized to Json
• Unsupported underlying type for producing LogicalType via co
• Unsupported field type: {type}
• AWS credential provider type '%s' is not supported
• Failed to access private field '%s' of AWS credential provid

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a957723ed943af8a. Report an issue: GitHub.