apache/beam · error · IllegalArgumentException

ApproximateUnique.PerKey needs an estimation error between 1

Error message

ApproximateUnique.PerKey needs an estimation error between 1% (0.01) and 50% (0.5).

What it means

ApproximateUnique.PerKey estimates the number of distinct values per key, and its accuracy is controlled by an estimation error parameter. The library throws this IllegalArgumentException when the supplied estimation error is below 0.01 (1%) or above 0.5 (50%), because the underlying K-Minimum-Values algorithm only gives meaningful guarantees within that range. Values outside the range would silently produce misleading sample sizes or wasted work, so the constructor rejects them eagerly.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateUnique.java:255

     */
    public PerKey(int sampleSize) {
      if (sampleSize < 16) {
        throw new IllegalArgumentException(
            "ApproximateUnique needs a "
                + "sampleSize >= 16 for an estimation error <= 50%.  In general, "
                + "the estimation error is about 2 / sqrt(sampleSize).");
      }

      this.sampleSize = sampleSize;
      this.maximumEstimationError = null;
    }

    /**
     * @see ApproximateUnique#perKey(double)
     */
    public PerKey(double estimationError) {
      if (estimationError < 0.01 || estimationError > 0.5) {
        throw new IllegalArgumentException(
            "ApproximateUnique.PerKey needs an "
                + "estimation error between 1% (0.01) and 50% (0.5).");
      }

      this.sampleSize = sampleSizeFromEstimationError(estimationError);
      this.maximumEstimationError = estimationError;
    }

    @Override
    public PCollection<KV<K, Long>> expand(PCollection<KV<K, V>> input) {
      Coder<KV<K, V>> inputCoder = input.getCoder();
      if (!(inputCoder instanceof KvCoder)) {
        throw new IllegalStateException(
            "ApproximateUnique.PerKey requires its input to use KvCoder");
      }
      @SuppressWarnings("unchecked")
      final Coder<V> coder = ((KvCoder<K, V>) inputCoder).getValueCoder();

View on GitHub (pinned to 12126d8942)

Solutions

1. Express the error as a decimal fraction between 0.01 and 0.5 (e.g. 5% -> 0.05)
2. Clamp the user-supplied value before constructing: Math.max(0.01, Math.min(0.5, err))
3. If you need tighter than 1% accuracy, use exact counting (Count.perKey / Distinct) instead of ApproximateUnique

Example fix

// before
PCollection<KV<K, Long>> counts = input.apply(ApproximateUnique.perKey(5.0));
// after
PCollection<KV<K, Long>> counts = input.apply(ApproximateUnique.perKey(0.05));

if (estimationError < 0.01 || estimationError > 0.5) {
  throw new IllegalArgumentException("estimationError must be in [0.01, 0.5], got " + estimationError);
}

When it happens

Trigger: Calling ApproximateUnique.perKey(double) or constructing new ApproximateUnique.PerKey<>(estimationError) with estimationError < 0.01 or estimationError > 0.5, including edge cases like 0.0 or 1.0.

Common situations: Developers copy a sampleSize-based setup and pass a percentage like 5 instead of 0.05, or ask for 0.1% precision (0.001) not realizing the algorithm cannot guarantee it.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Only %s objects with the same window supplier are compatible
• ApproximateUnique.PerKey requires its input to use KvCoder
• timerFrequency must be greater than zero
• Cannot provide coder for elements of Create: For their commo
• Cannot provide coder for elements of Create: For their commo

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2ac2d7733f80eb4e. Report an issue: GitHub.