apache/beam · error · IllegalArgumentException

smallBatchTag and largeBatchTag must not be null

Error message

smallBatchTag and largeBatchTag must not be null

What it means

BundleLifter's private constructor validates its inputs: the smallBatchTag and largeBatchTag TupleTags must be non-null (and distinct). A null tag would make the downstream partition/union operation unable to route elements, so it fails fast with IllegalArgumentException.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/BundleLifter.java:133

      OutputReceiver<T> taggedOutput = receiver.get(targetTag);

      for (T element : buffer) {
        taggedOutput.output(element);
      }
    }
  }

  private BundleLifter(TupleTag<T> smallBatchTag, TupleTag<T> largeBatchTag, int threshold) {
    this(smallBatchTag, largeBatchTag, threshold, x -> 1);
  }

  private BundleLifter(
      TupleTag<T> smallBatchTag,
      TupleTag<T> largeBatchTag,
      int threshold,
      SerializableFunction<T, Integer> elementSizer) {
    if (smallBatchTag == null || largeBatchTag == null) {
      throw new IllegalArgumentException("smallBatchTag and largeBatchTag must not be null");
    }
    if (smallBatchTag.getId().equals(largeBatchTag.getId())) {
      throw new IllegalArgumentException("smallBatchTag and largeBatchTag must be different");
    }
    if (threshold <= 0) {
      throw new IllegalArgumentException("Threshold must be a positive integer");
    }

    this.smallBatchTag = smallBatchTag;
    this.largeBatchTag = largeBatchTag;
    this.threshold = threshold;
    this.elementSizer = elementSizer;
  }

  public static <T> BundleLifter<T> of(
      TupleTag<T> smallBatchTag, TupleTag<T> largeBatchTag, int threshold) {
    return new BundleLifter<>(smallBatchTag, largeBatchTag, threshold);
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Provide distinct, non-null TupleTag instances for both small and large batches
2. Initialize the tags in the transform's builder/constructor before expansion
3. Assert tag non-nullness earlier (at builder build() time) for clearer errors
4. Ensure the two tags have different ids

Example fix

// before
new BundleLifter<>(null, largeTag, threshold, sizer);
// after
new BundleLifter<>(TupleTag["small"], TupleTag["large"], threshold, sizer);

Objects.requireNonNull(smallBatchTag, "smallBatchTag"); Objects.requireNonNull(largeBatchTag, "largeBatchTag");

boolean validTags(TupleTag<?> a, TupleTag<?> b){ return a!=null && b!=null && !a.getId().equals(b.getId()); }

try { lifter = newBundleLifter(small, large, threshold, sizer); } catch (IllegalArgumentException e) { throw new PipelineSetupException(e); }

When it happens

Trigger: Programmatically constructing a BundleLifter (e.g. builder or internal expansion of a batching transform) while passing null for either TupleTag.

Common situations: Custom Beam pipeline code reusing BundleLifter for bespoke batching; refactors where tags were conditionally omitted; misconfigured PTransform builders that skip tag initialization.

Related errors

• Expecting exactly one field, found
• %s expects a single %s tagged PCollection<Row> input
• Could not set a field in the BeamRowWrapper because rowData
• Unsupported Beam type for Iceberg timestamp with timezone: {
• Received null value for required field '{fieldName}'.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fa488fad809adeb7. Report an issue: GitHub.