apache/beam · error · IllegalArgumentException

Original transform had an output with tag but upgraded…

Error message

Original transform had an output with tag  but upgraded transform did not.

What it means

TransformUpgrader.updateTransformViaTransformService compares the output tags of the original PTransform against the upgraded (expanded) transform returned by the transform service. If any output tag present in the original transform's outputsMap is missing from the expanded transform's outputsMap, the upgrade is invalid and the pipeline would be corrupted, so an IllegalArgumentException is thrown.

Solutions

1. Ensure the upgraded transform definition preserves all output tags of the original transform
2. Pin the transform service / expansion service to a compatible version whose output contract matches the pipeline
3. Inspect the transform's outputsMap vs the expanded transform's outputsMap to find the removed/renamed tag and update the pipeline graph accordingly

Example fix

// before: upgraded transform drops tag 'out2'
// after: ensure expanded transform's OutputsMap contains every key of the original:
// original: {out1: pc1, out2: pc2}
// expanded must map: {out1: ..., out2: ...}

for (String tag : originalTransform.getOutputsMap().keySet()) {
  if (!expandedTransform.getOutputsMap().containsKey(tag)) {
    throw new IllegalStateException("Upgraded transform missing output tag: " + tag);
  }
}

try {
  upgrader.upgradeTransformsViaTransformService(pipeline, ids, service);
} catch (IllegalArgumentException e) {
  // handle missing output tag in upgraded transform
}

When it happens

Trigger: Calling TransformUpgrader.upgradeTransformsViaTransformService where a transform scheduled for upgrade declares an output PCollection tag that the upgraded transform definition no longer produces.

Common situations: The transform service returns a newer/different version of the transform whose output schema changed (a tag was renamed or removed); hand-edited or stale pipeline graph files referencing outdated transform outputs.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Could not find an output with tag for the transform
• Could not find a transform with id
• getClass() + " needs to override getOutputCoder()."
• A list of URNs for overriding transforms was provided but…
• A cannot be expanded

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7e6fb01394d6f887. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/TransformUpgrader.java:311

        UPGRADE_KEY, ByteString.copyFromUtf8(transformServiceVersion));
    expandedTransform = expandedTransformBuilder.build();

    List<String> expandedRequirements = response.getRequirementsList();

    RunnerApi.Components.Builder newComponentsBuilder = expandedComponents.toBuilder();

    // We record transforms that consume outputs of the old transform and update them to consume
    // outputs of the new (upgraded) transform.
    Collection<String> oldOutputs = transformToUpgrade.getOutputsMap().values();
    Map<String, String> inputReplacements = new HashMap<>();
    if (transformToUpgrade.getOutputsMap().size() == 1) {
      inputReplacements.put(
          oldOutputs.iterator().next(),
          expandedTransform.getOutputsMap().values().iterator().next());
    } else {
      for (Map.Entry<String, String> entry : transformToUpgrade.getOutputsMap().entrySet()) {
        if (!expandedTransform.getOutputsMap().keySet().contains(entry.getKey())) {
          throw new IllegalArgumentException(
              "Original transform had an output with tag "
                  + entry.getKey()
                  + " but upgraded transform did not.");
        }
        String newOutput = expandedTransform.getOutputsMap().get(entry.getKey());
        if (newOutput == null) {
          throw new IllegalArgumentException(
              "Could not find an output with tag "
                  + entry.getKey()
                  + " for the transform "
                  + expandedTransform);
        }
        inputReplacements.put(entry.getValue(), newOutput);
      }
    }

    // The list of obsolete (overridden) transforms that should be removed from the pipeline
    // produced by this method.

View on GitHub (pinned to 12126d8942)