{"record":{"id":"f85df2ad60abd93f","repo":"apache/hadoop","slug":"key-can-not-be-null","errorCode":null,"errorMessage":"key can not be null","messagePattern":"key can not be null","errorType":"exception","errorClass":"NullPointerException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/CountingBloomFilter.java","lineNumber":107,"sourceCode":"   * @param nbHash The number of hash function to consider.\n   * @param hashType type of the hashing function (see\n   * {@link org.apache.hadoop.util.hash.Hash}).\n   */\n  public CountingBloomFilter(int vectorSize, int nbHash, int hashType) {\n    super(vectorSize, nbHash, hashType);\n    buckets = new long[buckets2words(vectorSize)];\n  }\n\n  /** returns the number of 64 bit words it would take to hold vectorSize buckets */\n  private static int buckets2words(int vectorSize) {\n   return ((vectorSize - 1) >>> 4) + 1;\n  }\n\n\n  @Override\n  public void add(Key key) {\n    if(key == null) {\n      throw new NullPointerException(\"key can not be null\");\n    }\n\n    int[] h = hash.hash(key);\n    hash.clear();\n\n    for(int i = 0; i < nbHash; i++) {\n      // find the bucket\n      int wordNum = h[i] >> 4;          // div 16\n      int bucketShift = (h[i] & 0x0f) << 2;  // (mod 16) * 4\n      \n      long bucketMask = 15L << bucketShift;\n      long bucketValue = (buckets[wordNum] & bucketMask) >>> bucketShift;\n      \n      // only increment if the count in the bucket is less than BUCKET_MAX_VALUE\n      if(bucketValue < BUCKET_MAX_VALUE) {\n        // increment by 1\n        buckets[wordNum] = (buckets[wordNum] & ~bucketMask) | ((bucketValue + 1) << bucketShift);\n      }","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/CountingBloomFilter.java#L89-L125","documentation":"CountingBloomFilter.add(Key) hashes the key and increments 4-bit counters in its long[] buckets array; a null key cannot be hashed and the method throws NullPointerException('key can not be null') as an explicit precondition (note the wording differs slightly from BloomFilter's 'key cannot be null'). Fail-fast up front avoids a JVM NPE from inside the hash implementation.","triggerScenarios":"countingBloomFilter.add(null); adding keys pulled from a nullable source (map.get on absent key, sparsely populated array); pipelines where the key extractor returns null for malformed records.","commonSituations":"Frequency-count or duplicate-detection use of counting bloom filters on dirty data; record parsing where one field being absent yields a null Key; concurrent maps returning null under eviction races.","solutions":["Null-check the Key before add() and skip/reject the record.","Fix the producer of nulls (defaults, Optional, filter(Objects::nonNull)).","Add a boundary requireNonNull at data ingestion so failures name the real culprit."],"exampleFix":"// before\ncbf.add(parseKey(line)); // parseKey returned null for a malformed line\n// throws NullPointerException: key can not be null\n\n// after\nKey k = parseKey(line);\nif (k != null) {\n  cbf.add(k);\n}","handlingStrategy":"validation","validationCode":"Key k = parseKey(record);\nif (k == null) {\n  metrics.incrMalformed();\n  return;\n}\ncbf.add(k);","typeGuard":null,"tryCatchPattern":"try {\n  cbf.add(key);\n} catch (NullPointerException e) {\n  LOG.warn(\"Rejected null key for counting bloom filter\", e);\n}","preventionTips":["Validate records at ingestion so Key producers never emit null.","Use Objects.requireNonNull at data boundaries with descriptive messages.","Add null-tolerant loops (if (k != null)) for sweeps over sparse collections."],"tags":["bloom-filter","counting-bloom-filter","null-check","precondition","hadoop-common"],"backgroundTag":"null-argument","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}