apache/beam · error · IllegalArgumentException

ApproximateUnique needs a sampleSize >= 16 for an…

Error message

ApproximateUnique needs a sampleSize >= 16 for an estimation error <= 50%.  In general, the estimation error is about 2 / sqrt(sampleSize).

What it means

ApproximateUnique.Globally requires sampleSize >= 16 because the HyperLogLog-based estimator's error grows as ~2/sqrt(sampleSize); below 16 the promised error bound is meaningless, so the constructor throws IllegalArgumentException.

Solutions

1. Set sampleSize to at least 16 (choose based on 2/sqrt(sampleSize) error tolerance)
2. Use the ApproximateUnique.globally(double error) overload to derive sampleSize from a desired error
3. Clamp user-provided sample sizes: Math.max(16, sampleSize)

Example fix

// before
ApproximateUnique.globally(10)
// after
ApproximateUnique.globally(16)  // or ApproximateUnique.globally(0.05)

if (sampleSize < 16) throw new IllegalArgumentException("sampleSize must be >= 16 for ApproximateUnique");

try { ApproximateUnique.globally(n); }
catch (IllegalArgumentException e) { /* clamp and retry: */ p = p.apply(ApproximateUnique.globally(16)); }

When it happens

Trigger: Constructing ApproximateUnique.Globally with a sample size of 15 or less, e.g. via new ApproximateUnique.Globally(8) or ApproximateUnique.globally(n) with n < 16.

Common situations: Developers tune sampleSize down for performance without realizing the minimum, or pass a placeholder 0/1 value while scaffolding a pipeline.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• ApproximateUnique needs an estimation error between 1%…
• Expecting exactly one field, found
• Initial capacity < 0
• Invalid Firestore document name
• Iterable at position

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/eba2cf48cc3a6041. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateUnique.java:179

   * @param <T> the type of the elements in the input {@code PCollection}
   */
  public static final class Globally<T> extends PTransform<PCollection<T>, PCollection<Long>> {

    /**
     * The number of entries in the statistical sample; the higher this number, the more accurate
     * the estimate will be.
     */
    private final long sampleSize;

    /** The desired maximum estimation error or null if not specified. */
    private final @Nullable Double maximumEstimationError;

    /**
     * @see ApproximateUnique#globally(int)
     */
    public Globally(int sampleSize) {
      if (sampleSize < 16) {
        throw new IllegalArgumentException(
            "ApproximateUnique needs a sampleSize "
                + ">= 16 for an estimation error <= 50%.  "
                + "In general, the estimation "
                + "error is about 2 / sqrt(sampleSize).");
      }

      this.sampleSize = sampleSize;
      this.maximumEstimationError = null;
    }

    /**
     * @see ApproximateUnique#globally(double)
     */
    public Globally(double maximumEstimationError) {
      if (maximumEstimationError < 0.01 || maximumEstimationError > 0.5) {
        throw new IllegalArgumentException(
            "ApproximateUnique needs an " + "estimation error between 1% (0.01) and 50% (0.5).");
      }

View on GitHub (pinned to 12126d8942)