apache/beam · error · java.lang.IllegalArgumentException

Duplicate values for

Error message

Duplicate values for <key>

What it means

While flattening the multimap materialization into a Map<K,V>, the view apply found the same key more than once among the KV entries. Map views require unique keys; the duplicate <key> entries in the input multimap are at fault (this remains a known issue around Java vs structural equality, per the adjacent TODO).

Solutions

  1. Copy into a new ArrayList and call remove on the copy.
  2. Use stream filter to build a list without the unwanted element (or by index: IntStream.range filtering).
  3. Remove the elements upstream with a Beam Filter transform before creating the view.

Example fix

// before
sideInputList.remove(i);
// after
List<T> mutable = new ArrayList<>(sideInputList);
mutable.remove(i);
Defensive patterns

Strategy: fallback

Try / catch

try {
  viewedList.remove(i);
} catch (UnsupportedOperationException e) {
  List<T> copy = new ArrayList<>(viewedList);
  copy.remove(i);
  viewedList = copy;
}

Prevention

When it happens

Trigger: Calling list.remove(i) on the List obtained from a Beam side input view.

Common situations: Deleting an entry from a side input list during element processing; refactoring old ArrayList-based code to use Beam views without adjusting mutability assumptions.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d2c7ed504425c9d2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/values/PCollectionViews.java:1627

        TypeDescriptorSupplier<V> valueTypeDescriptorSupplier) {
      this.keyTypeDescriptorSupplier = keyTypeDescriptorSupplier;
      this.valueTypeDescriptorSupplier = valueTypeDescriptorSupplier;
    }

    @Override
    public Materialization<MultimapView<Void, KV<K, V>>> getMaterialization() {
      return Materializations.multimap();
    }

    @Override
    public Map<K, V> apply(MultimapView<Void, KV<K, V>> primitiveViewT) {
      // TODO: https://github.com/apache/beam/issues/18569 - fix this so that we aren't relying on
      // Java equality and are
      // using structural value equality.
      Map<K, V> map = new HashMap<>();
      for (KV<K, V> elem : primitiveViewT.get(null)) {
        if (map.containsKey(elem.getKey())) {
          throw new IllegalArgumentException("Duplicate values for " + elem.getKey());
        }
        map.put(elem.getKey(), elem.getValue());
      }
      return Collections.unmodifiableMap(map);
    }

    @Override
    public TypeDescriptor<Map<K, V>> getTypeDescriptor() {
      return TypeDescriptors.maps(
          keyTypeDescriptorSupplier.get(), valueTypeDescriptorSupplier.get());
    }
  }

  /**
   * Implementation which is able to adapt an iterable materialization to an in-memory {@code Map<K,
   * V>}.
   *
   * <p>For internal use only.

View on GitHub (pinned to 12126d8942)