apache/beam · error · IllegalArgumentException

Unknown %s type %s

Error message

Unknown %s type %s

What it means

ReadTranslation.toProto(Source<?>) converts an org.apache.beam.sdk.io.Source into its protobuf FunctionSpec representation. Only BoundedSource and UnboundedSource subclasses are supported; any other Source implementation triggers an IllegalArgumentException with 'Unknown Source type <class>'. It exists because the wire protocol only defines payload translators for those two source kinds.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/ReadTranslation.java:75

        .setIsBounded(IsBounded.Enum.BOUNDED)
        .setSource(toProto(read.getSource()))
        .build();
  }

  public static ReadPayload toProto(SplittableParDo.PrimitiveUnboundedRead<?> read) {
    return ReadPayload.newBuilder()
        .setIsBounded(IsBounded.Enum.UNBOUNDED)
        .setSource(toProto(read.getSource()))
        .build();
  }

  public static FunctionSpec toProto(Source<?> source) {
    if (source instanceof BoundedSource) {
      return toProto((BoundedSource) source);
    } else if (source instanceof UnboundedSource) {
      return toProto((UnboundedSource<?, ?>) source);
    } else {
      throw new IllegalArgumentException(
          String.format("Unknown %s type %s", Source.class.getSimpleName(), source.getClass()));
    }
  }

  private static FunctionSpec toProto(BoundedSource<?> source) {
    return FunctionSpec.newBuilder()
        .setUrn(JAVA_SERIALIZED_BOUNDED_SOURCE)
        .setPayload(ByteString.copyFrom(SerializableUtils.serializeToByteArray(source)))
        .build();
  }

  public static BoundedSource<?> boundedSourceFromProto(ReadPayload payload)
      throws InvalidProtocolBufferException {
    checkArgument(payload.getIsBounded().equals(IsBounded.Enum.BOUNDED));
    return (BoundedSource<?>)
        SerializableUtils.deserializeFromByteArray(
            payload.getSource().getPayload().toByteArray(), "BoundedSource");
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Make the custom source extend BoundedSource (or UnboundedSource) instead of raw Source.
2. Use a built-in Beam connector (e.g. the newer splittable DoFn-based APIs) that supports proto translation.
3. If you own the translation path, register a payload translator for the custom source type.
4. Check the concrete class in the message and confirm which interface it implements.

Example fix

// before
class MySource extends Source<String> { ... }
// after
class MySource extends BoundedSource<String> {
  @Override public Coder<String> getOutputCoder() { ... }
  @Override public List<? extends BoundedSource<String>> split(long desiredBundleSizeBytes, PipelineOptions options) { ... }
}

if (!(source instanceof BoundedSource) && !(source instanceof UnboundedSource)) { throw new IllegalArgumentException("Unsupported source: " + source.getClass()); }

boolean isTranslatable(Source<?> s) { return s instanceof BoundedSource || s instanceof UnboundedSource; }

try { FunctionSpec spec = ReadTranslation.toProto(source); } catch (IllegalArgumentException e) { /* route to custom translator */ }

When it happens

Trigger: Calling ReadTranslation.toProto(source) with a Source that is neither a BoundedSource nor an UnboundedSource — e.g. a custom Source subclass that directly extends Source, or a Source returned by a third-party connector.

Common situations: Custom IO connectors written against the abstract Source base class instead of BoundedSource/UnboundedSource, using legacy non-splittable sources with runners that translate pipelines to proto, or version changes where a source class no longer extends the expected type.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Unable to provide coder for %s, this factory can only provid
• getClass() + " needs to override getOutputCoder()."
• Need to set the filepattern of a TFRecordIO.Read transform
• expected %d, but got %d
• The input schema must have exactly one field of type byte.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/cc627e60874ee687. Report an issue: GitHub.