apache/beam · error · IllegalArgumentException

can not be serialized to Json

Error message

 can not be serialized to Json

What it means

AwsSerializableUtils.serialize uses Jackson MAPPER.writeValueAsString; if Jackson raises JsonProcessingException it rethrows IllegalArgumentException with '<ClassName> can not be serialized to Json'. The object (typically an AwsCredentialsProvider) lacks Jackson-serializable structure or a working serializer for its type.

Source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/options/AwsSerializableUtils.java:48

  static {
    MAPPER.registerModule(new AwsModule());
  }

  public static String serializeAwsCredentialsProvider(AwsCredentialsProvider credentialsProvider) {
    return serialize(credentialsProvider);
  }

  public static AwsCredentialsProvider deserializeAwsCredentialsProvider(
      String serializedCredentialsProvider) {
    return deserialize(serializedCredentialsProvider, AwsCredentialsProvider.class);
  }

  static String serialize(Object object) {
    try {
      return MAPPER.writeValueAsString(object);
    } catch (JsonProcessingException e) {
      throw new IllegalArgumentException(
          object.getClass().getSimpleName() + " can not be serialized to Json", e);
    }
  }

  static <T> T deserialize(String serializedObject, Class<T> clazz) {
    try {
      return MAPPER.readValue(serializedObject, clazz);
    } catch (IOException e) {
      throw new IllegalArgumentException(
          clazz.getSimpleName() + " can not be deserialized from Json", e);
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Use a Beam-supported credentials provider type that has registered Jackson mixins (see AwsModule).
2. If serializing a custom type, register a Jackson serializer/mixin for it or expose only serializable properties.
3. Remove non-serializable fields (clients, connections, lambdas) from the object being serialized.

Example fix

// before
options.setAwsCredentialsProvider(new MyProvider(kinesisClient)); // holds a client
// after
options.setAwsCredentialsProvider(DefaultCredentialsProvider.create());

try {
  String json = AwsSerializableUtils.serializeAwsCredentialsProvider(provider);
} catch (IllegalArgumentException e) {
  // provider is not Jackson-serializable; substitute a supported provider
}

When it happens

Trigger: Calling serializeAwsCredentialsProvider (or serialize) on an object Jackson cannot write — e.g. a custom provider without registered mixin/serializer, containing non-serializable fields like streams, lambdas or clients.

Common situations: Passing a custom AwsCredentialsProvider into AwsOptions which is then serialized for job submission; provider holding an SDK client or IO resource that Jackson cannot marshal.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• AWS credential provider type '%s' is not supported
• Unsupported AWS credentials provider type
• Failed to serialize and deserialize property '%s' with value
• Failed to convert PipelineOptions to Protocol
• Failed to serialize %s value: %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/df3723bae7108225. Report an issue: GitHub.