apache/beam · error · IllegalArgumentException

PCollections come from different Pipelines

Error message

PCollections come from different Pipelines

What it means

PCollectionRowTuple.and(String, PCollection<Row>) requires the added PCollection to belong to the tuple's Pipeline; on mismatch it throws IllegalArgumentException because Beam cannot mix PCollections from different Pipelines in one tuple.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/values/PCollectionRowTuple.java:151

      PCollection<Row> pc5) {
    return of(tag1, pc1, tag2, pc2, tag3, pc3, tag4, pc4).and(tag5, pc5);
  }

  // To create a PCollectionRowTuple with more than five inputs, use the and() builder method.

  /**
   * Returns a new {@link PCollectionRowTuple} that has each {@link PCollection} and tag of this
   * {@link PCollectionRowTuple} plus the given {@link PCollection} associated with the given tag.
   *
   * <p>The given tag should not already be mapped to a {@link PCollection} in this {@link
   * PCollectionRowTuple}.
   *
   * <p>Each {@link PCollection} in the resulting {@link PCollectionRowTuple} must be part of the
   * same {@link Pipeline}.
   */
  public PCollectionRowTuple and(String tag, PCollection<Row> pc) {
    if (pc.getPipeline() != pipeline) {
      throw new IllegalArgumentException("PCollections come from different Pipelines");
    }

    return new PCollectionRowTuple(
        pipeline,
        new ImmutableMap.Builder<String, PCollection<Row>>()
            .putAll(pcollectionMap)
            .put(tag, pc)
            .build());
  }

  /**
   * Returns whether this {@link PCollectionRowTuple} contains a {@link PCollection} with the given
   * tag.
   */
  public boolean has(String tag) {
    return pcollectionMap.containsKey(tag);
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the PCollection was created by the same Pipeline as the tuple
2. Pass the tuple's Pipeline into the code that produces the PCollection<Row>
3. Rebuild the tuple entirely from PCollections of one Pipeline

Example fix

// before
tuple.and("rows", pcFromOtherPipeline); // throws
// after
PCollection<Row> rows = tuple.getPipeline().apply(...);
tuple.and("rows", rows);

if (pc.getPipeline() != tuple.getPipeline()) { throw new IllegalArgumentException("wrong pipeline"); }

boolean samePipeline(PCollection<?> pc, PCollectionRowTuple t) { return pc.getPipeline() == t.getPipeline(); }

try { return tuple.and(tag, pc); } catch (IllegalArgumentException e) { /* rebuild pc on tuple's pipeline */ }

When it happens

Trigger: Adding a PCollection<Row> produced by a different Pipeline instance to an existing PCollectionRowTuple via and(tag, pc).

Common situations: Merging Row outputs from separately built Pipelines (e.g. BigQuery reads in different test setups); creating a new Pipeline inside a helper that contributes to an existing tuple.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• PCollections come from different Pipelines
• PCollections come from different Pipelines
• The pipeline has not been run.
• The pipeline contains abandoned PTransform(s).
• ApproximateUnique.PerKey needs an estimation error between 1

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3f1e0a71ed02adc5. Report an issue: GitHub.