apache/beam · error · java.lang.RuntimeException

Did not find a SchemaTransformProvider with the identifier

Error message

Did not find a SchemaTransformProvider with the identifier 

What it means

ExpansionServiceSchemaTransformProvider.getTransform parses the FunctionSpec payload as a SchemaTransformPayload and looks up the identifier in its provider map. If the identifier is not registered, it throws a RuntimeException with this message. It means the expansion service cannot serve the requested SchemaTransform URN.

Solutions

1. Add the jar containing the missing SchemaTransformProvider to the expansion service classpath and restart it
2. Align Beam SDK versions between the client pipeline and the expansion service so identifier versions match
3. Print available identifiers (e.g., via getAllProviders()) and compare with the requested identifier to catch typos or version suffix mismatches

Example fix

// before: service launched without the JDBC io jar
java -jar expansion-service.jar
// after
java -cp expansion-service.jar:beam-sdks-java-io-jdbc-2.xx.0.jar org.apache.beam.sdk.expansion.service.ExpansionService ...

try { expansionService.expand(request); } catch (RuntimeException e) { if (e.getMessage().startsWith("Did not find a SchemaTransformProvider")) { /* add provider jar / fix identifier */ } throw e; }

When it happens

Trigger: An expansion request with URN SCHEMA_TRANSFORM whose payload identifier (e.g., 'beam:schematransform:org.apache.beam:my_transform:v1') has no matching SchemaTransformProvider on the service's classpath.

Common situations: Client SDK and expansion service at different Beam versions so the identifier/version string differs; provider jar not deployed to the service's classpath; identifier typo in pipeline construction.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• A list of URNs for overriding transforms was provided but…
• A cannot be expanded
• A transform cannot be initiated using the provided config…
• AVRO schema doesn't match row schema. Row schema
• BigQuery data contained value

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/af7be7c59047b573. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionServiceSchemaTransformProvider.java:100

  }

  @Override
  public Map<String, PCollection<?>> extractOutputs(PCollectionRowTuple output) {
    ImmutableMap.Builder<String, PCollection<?>> pCollectionMap = ImmutableMap.builder();
    for (String key : output.getAll().keySet()) {
      pCollectionMap.put(key, output.get(key));
    }
    return pCollectionMap.build();
  }

  @Override
  public PTransform getTransform(FunctionSpec spec, PipelineOptions options) {
    SchemaTransformPayload payload;
    try {
      payload = SchemaTransformPayload.parseFrom(spec.getPayload());
      String identifier = payload.getIdentifier();
      if (!schemaTransformProviders.containsKey(identifier)) {
        throw new RuntimeException(
            "Did not find a SchemaTransformProvider with the identifier " + identifier);
      }

    } catch (InvalidProtocolBufferException e) {
      throw new IllegalArgumentException(
          "Invalid payload type for URN " + getUrn(ExpansionMethods.Enum.SCHEMA_TRANSFORM), e);
    }

    String identifier = payload.getIdentifier();
    org.apache.beam.sdk.schemas.transforms.SchemaTransformProvider provider =
        schemaTransformProviders.get(identifier);
    if (provider == null) {
      throw new IllegalArgumentException(
          "Could not find a SchemaTransform with identifier " + identifier);
    }

    Schema configSchemaFromRequest =
        SchemaTranslation.schemaFromProto(payload.getConfigurationSchema());

View on GitHub (pinned to 12126d8942)