apache/beam · error · IllegalArgumentException

Could not find a transform with id

Error message

Could not find a transform with id 

What it means

recursivelyFindSubTransforms walks a transform's subtransform tree in the pipeline's Components. When a transform id (root or nested) is not present in components.getTransformsMap(), the graph is inconsistent and an IllegalArgumentException is thrown.

Solutions

1. Verify the transform id exists in the pipeline's Components before upgrading (components.getTransformsMap().containsKey(id))
2. Regenerate the pipeline graph from a supported SDK path rather than editing protos manually
3. Fix the parent transform's subtransforms list to only reference existing transform ids

Example fix

// before
recursivelyFindSubTransforms("missing_transform", components, results);
// after
if (components.getTransformsMap().containsKey("missing_transform")) {
  recursivelyFindSubTransforms("missing_transform", components, results);
}

if (!components.getTransformsMap().containsKey(transformId)) {
  throw new IllegalStateException("Transform id not in components: " + transformId);
}

try {
  upgrader.upgradeTransformsViaTransformService(pipeline, ids, service);
} catch (IllegalArgumentException e) {
  // handle missing transform id in components
}

When it happens

Trigger: Calling updateTransformViaTransformService / recursivelyFindSubTransforms with a transformId (or any subtransform id listed in a PTransform's subtransforms) that does not exist in RunnerApi.Components.transforms.

Common situations: Corrupted or hand-edited pipeline proto files; a transform referencing subtransforms that were pruned or never added to components; mismatched component IDs from an external pipeline exporter.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• Could not find an output with tag for the transform
• getClass() + " needs to override getOutputCoder()."
• Original transform had an output with tag but upgraded…
• A list of URNs for overriding transforms was provided but…
• A cannot be expanded

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/855a627edfdd5d83. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/TransformUpgrader.java:381

    newComponentsBuilder.clearTransforms();
    newComponentsBuilder.putAllTransforms(updatedExpandedTransformMap);
    newComponentsBuilder.putTransforms(transformId, expandedTransform);

    RunnerApi.Pipeline.Builder newRunnerAPIPipelineBuilder = runnerAPIpipeline.toBuilder();
    newRunnerAPIPipelineBuilder.clearComponents();
    newRunnerAPIPipelineBuilder.setComponents(newComponentsBuilder.build());

    newRunnerAPIPipelineBuilder.addAllRequirements(expandedRequirements);

    return newRunnerAPIPipelineBuilder.build();
  }

  private static void recursivelyFindSubTransforms(
      String transformId, RunnerApi.Components components, List<String> results) {
    results.add(transformId);
    RunnerApi.PTransform transform = components.getTransformsMap().get(transformId);
    if (transform == null) {
      throw new IllegalArgumentException("Could not find a transform with id " + transformId);
    }
    List<String> subTransforms = transform.getSubtransformsList();
    if (subTransforms != null) {
      for (String subTransformId : subTransforms) {
        recursivelyFindSubTransforms(subTransformId, components, results);
      }
    }
  }

  private static int findAvailablePort() throws IOException {
    ServerSocket s = new ServerSocket(0);
    try {
      return s.getLocalPort();
    } finally {
      s.close();
      try {
        // Some systems don't free the port for future use immediately.
        Thread.sleep(100);

View on GitHub (pinned to 12126d8942)