apache/hadoop · error · IllegalArgumentException

Key is not a member

Error message

Key is not a member

What it means

CountingBloomFilter.delete(Key) first calls membershipTest(key); if the filter does not currently report the key as a member, it throws IllegalArgumentException('Key is not a member'). Deleting a non-member would decrement counters that other keys rely on, corrupting the filter, so it is forbidden. (Note the javadoc above the method claims 'nothing happens' for non-members — the code actually throws, so trust the throw.)

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/CountingBloomFilter.java:140

      if(bucketValue < BUCKET_MAX_VALUE) {
        // increment by 1
        buckets[wordNum] = (buckets[wordNum] & ~bucketMask) | ((bucketValue + 1) << bucketShift);
      }
    }
  }

  /**
   * Removes a specified key from <i>this</i> counting Bloom filter.
   * <p>
   * <b>Invariant</b>: nothing happens if the specified key does not belong to <i>this</i> counter Bloom filter.
   * @param key The key to remove.
   */
  public void delete(Key key) {
    if(key == null) {
      throw new NullPointerException("Key may not be null");
    }
    if(!membershipTest(key)) {
      throw new IllegalArgumentException("Key is not a member");
    }

    int[] h = hash.hash(key);
    hash.clear();

    for(int i = 0; i < nbHash; i++) {
      // find the bucket
      int wordNum = h[i] >> 4;          // div 16
      int bucketShift = (h[i] & 0x0f) << 2;  // (mod 16) * 4
      
      long bucketMask = 15L << bucketShift;
      long bucketValue = (buckets[wordNum] & bucketMask) >>> bucketShift;
      
      // only decrement if the count in the bucket is between 0 and BUCKET_MAX_VALUE
      if(bucketValue >= 1 && bucketValue < BUCKET_MAX_VALUE) {
        // decrement by 1
        buckets[wordNum] = (buckets[wordNum] & ~bucketMask) | ((bucketValue - 1) << bucketShift);
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Only delete keys you have previously added with the same filter instance.
2. Guard with if (cbf.membershipTest(key)) cbf.delete(key);
3. Make removal idempotent: mark keys processed (e.g. a secondary set) so double-deletes cannot occur.
4. Never delete after shape-altering operations (and/xor) unless you know membership still holds; rebuild instead.

Example fix

// before
cbf.delete(key); // key was never added (or already deleted)
// throws IllegalArgumentException: Key is not a member

// after
if (cbf.membershipTest(key)) {
  cbf.delete(key);
}

// idempotent delete: only remove keys the filter still reports
if (cbf.membershipTest(key)) {
  cbf.delete(key);
} else {
  LOG.debug("Key not present; nothing to delete");
}

try {
  cbf.delete(key);
} catch (IllegalArgumentException e) {
  // 'Key is not a member': already deleted or never added — treat as no-op
  LOG.debug("Ignored delete of non-member key", e);
}

When it happens

Trigger: cbf.delete(key) for a key that was never added; deleting a key that was already deleted once (second delete fails membership); deleting after and()-ing the filter with another filter wiped the counters; deleting a key whose add() was rolled back by an exception mid-loop.

Common situations: Remove-on-expiry loops where the same key is processed twice; sets rebuilt from serialized filters where prior adds were lost; compensating transactions that delete records that were never inserted; the (rare) false-positive direction does not cause this — only false negatives (from filter corruption or mismatched shape) make an added key test as non-member.

Related errors

• key can not be null
• Key may not be null
• key cannot be null
• filters cannot be and-ed
• Value length unknown.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c8f7fcaee05aaa4c. Report an issue: GitHub.