apache/beam · error · IllegalArgumentException

Expected a single PCollection output, but got

Error message

Expected a single PCollection output, but got ${output}. Perhaps withMultipleOutputs() needs to be specified?

What it means

When the YAML transform declares no multiple output tags, expand() expects the underlying external transform to produce exactly one PCollection. If the returned value is a multi-output PCollectionTuple/PCollectionRowTuple instead, the library throws, hinting that the transform was configured for multiple outputs.

Solutions

1. Declare the multiple output tags in the YAML/Java config (set outputTags) so multi-output is expected.
2. Configure the external transform for a single output if only one result is needed.
3. Update the pipeline consumer to handle the PCollectionRowTuple of outputs explicitly.
4. Check the external transform's withMultipleOutputs usage and remove it for single-output usage.

Example fix

// before (multi-output transform, single-output spec)
yamlTransform.withOutputTags(null);
// after
yamlTransform.withOutputTags(new TupleTag<>("main"), ImmutableList.of(new TupleTag<>("side")));

if (specOutputTags == null && !(externalTransformSupportsSingleOutput())) {
  throw new IllegalStateException("Declare outputTags for multi-output transform");
}

boolean isSingleOutput(Object output) { return output instanceof PCollection; }

try {
  return expand(input);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("withMultipleOutputs")) { /* declare output tags */ }
  throw e;
}

When it happens

Trigger: Applying a YamlTransform whose underlying external (e.g. Python) transform returns multiple outputs (via withMultipleOutputs) while the Java-side outputTags spec is null/absent.

Common situations: YAML spec drift: the external transform was changed to emit several named outputs but the YAML/Java wrapper still declares a single output; copy-pasting a multi-output transform config into a single-output pipeline.

Related errors

• Expected outputs, but got exactly one.
• Input has tags but expected input tags
• Output has tags but expected output tags
• Unrecognized input type
• A function must be provided to convert the input type into…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a50601c3dcef3f8c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/yaml/src/main/java/org/apache/beam/sdk/extensions/yaml/YamlTransform.java:175

    PTransform externalTransform =
        PythonExternalTransform.from("apache_beam.yaml.yaml_transform.YamlTransform")
            .withArgs(yamlDefinition)
            .withExtraPackages(ImmutableList.of("jinja2", "pyyaml", "virtualenv-clone"));
    if (input instanceof PBegin) {
      output = ((PBegin) input).apply(externalTransform);
    } else if (input instanceof PCollection) {
      output = ((PCollection<?>) input).apply(externalTransform);
    } else if (input instanceof PCollection) {
      output = ((PCollection<?>) input).apply(externalTransform);
    } else if (input instanceof PCollectionRowTuple) {
      output = ((PCollectionRowTuple) input).apply(externalTransform);
    } else {
      throw new IllegalArgumentException("Unrecognized input type: " + input);
    }

    if (outputTags == null) {
      if (!(output instanceof PCollection)) {
        throw new IllegalArgumentException(
            "Expected a single PCollection output, but got "
                + output
                + ". Perhaps withMultipleOutputs() needs to be specified?");
      }
      return (OutputT) output;
    } else {
      if (output instanceof PCollection) {
        // ExternalPythonTransform always returns single outputs as PCollections.
        if (outputTags.size() != 1) {
          throw new IllegalArgumentException(
              "Expected " + outputTags.size() + " outputs, but got exactly one.");
        }
        return (OutputT)
            PCollectionRowTuple.of(outputTags.iterator().next(), (PCollection<Row>) output);
      } else {
        Map<TupleTag<?>, PValue> expandedOutputs = output.expand();
        Set<String> actualOutputTags =
            expandedOutputs.keySet().stream()

View on GitHub (pinned to 12126d8942)