What causes "key cannot be null" in apache/hadoop?

Null-check the key before calling add() and skip or reject the record.

apache/hadoop · error · NullPointerException

key cannot be null

Error message

key cannot be null

What it means

BloomFilter.add(Key) hashes the key and sets the corresponding bits in its BitSet; a null key cannot be hashed, so the method throws NullPointerException with this message as an explicit precondition check. This is fail-fast validation, not a JVM-generated NPE — the class rejects null up front with a clear message.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/BloomFilter.java:119

  }
  
  /**
   * Constructor
   * @param vectorSize The vector size of <i>this</i> filter.
   * @param nbHash The number of hash function to consider.
   * @param hashType type of the hashing function (see
   * {@link org.apache.hadoop.util.hash.Hash}).
   */
  public BloomFilter(int vectorSize, int nbHash, int hashType) {
    super(vectorSize, nbHash, hashType);

    bits = new BitSet(this.vectorSize);
  }

  @Override
  public void add(Key key) {
    if(key == null) {
      throw new NullPointerException("key cannot be null");
    }

    int[] h = hash.hash(key);
    hash.clear();

    for(int i = 0; i < nbHash; i++) {
      bits.set(h[i]);
    }
  }

  @Override
  public void and(Filter filter) {
    if(filter == null
        || !(filter instanceof BloomFilter)
        || filter.vectorSize != this.vectorSize
        || filter.nbHash != this.nbHash) {
      throw new IllegalArgumentException("filters cannot be and-ed");
    }

View on GitHub (pinned to 2add963021)

Solutions

Null-check the key before calling add() and skip or reject the record.
Fix the upstream source of nulls (map.get default, Optional, filter(Objects::nonNull)).
Wrap the call in try/catch NullPointerException if you cannot change the producer (last resort).

Example fix

// before
bloomFilter.add(keyMap.get(id)); // get(id) returned null
// throws NullPointerException: key cannot be null

// after
Key k = keyMap.get(id);
if (k != null) {
  bloomFilter.add(k);
}

Defensive patterns

Strategy: validation

Validate before calling

if (key == null) {
  LOG.debug("Skipping null key for bloom filter");
  return;
}
bloomFilter.add(key);

Try / catch

try {
  bloomFilter.add(key);
} catch (NullPointerException e) {
  // guard against nulls from untrusted producers
  LOG.warn("Rejected null key", e);
}

Prevention

Filter nulls at ingestion: stream.filter(Objects::nonNull).
Use Optional or containsKey-style checks instead of nullable key lookups.
Wrap external input parsing with a non-null Key factory that throws with context.

When it happens

Trigger: bloomFilter.add(null); adding keys from a map lookup that returned null (get on a missing key); stream pipelines feeding nulls into the filter; test code constructing Key arrays with gaps.

Common situations: Cache-dedup or previously-seen checks where the key source can be absent; deserialization producing null Keys for missing fields; refactoring that changed a key extractor to return null.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f2714b34b3253f0c. Report an issue: GitHub.