{"record":{"id":"e4d81636f399e529","repo":"apache/hadoop","slug":"key-is-not-a-member-e4d816","errorCode":null,"errorMessage":"Key is not a member","messagePattern":"Key is not a member","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/RetouchedBloomFilter.java","lineNumber":205,"sourceCode":"    }\n\n    for (int i = 0; i < keys.length; i++) {\n      addFalsePositive(keys[i]);\n    }\n  }\n\n  /**\n   * Performs the selective clearing for a given key.\n   * @param k The false positive key to remove from <i>this</i> retouched Bloom filter.\n   * @param scheme The selective clearing scheme to apply.\n   */\n  public void selectiveClearing(Key k, short scheme) {\n    if (k == null) {\n      throw new NullPointerException(\"Key can not be null\");\n    }\n\n    if (!membershipTest(k)) {\n      throw new IllegalArgumentException(\"Key is not a member\");\n    }\n\n    int index = 0;\n    int[] h = hash.hash(k);\n\n    switch(scheme) {\n\n    case RANDOM:\n      index = randomRemove();\n      break;\n    \n    case MINIMUM_FN:\n      index = minimumFnRemove(h);\n      break;\n    \n    case MAXIMUM_FP:\n      index = maximumFpRemove(h);\n      break;","sourceCodeStart":187,"sourceCodeEnd":223,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/RetouchedBloomFilter.java#L187-L223","documentation":"selectiveClearing first runs membershipTest(k); a key whose bits are not all set cannot be retouched, so the call is refused with IllegalArgumentException. This guards the filter invariant that you may only remove keys the filter currently claims as members (true members and false positives both pass that test).","triggerScenarios":"Calling selectiveClearing for a key never added via add(); a key whose bits were already cleared by an earlier retouch (now a false negative); a key re-encoded differently (re-serialized byte[]) than at add() time; replaying an old false-positive list against a rebuilt filter.","commonSituations":"Retouch pipelines feeding candidate keys from external storage without verifying membership; schema/encoding drift changing the bytes used to build the Key; regenerating the filter but reusing stale false-positive logs.","solutions":["Call rbf.membershipTest(k) first and skip non-members","Verify the Key bytes are byte-identical to those used at add() time (same encoding, same hash type)","Regenerate the false-positive list from the current filter generation instead of reusing an old one"],"exampleFix":"// before\nrbf.selectiveClearing(fpKey, RemoveScheme.RATIO); // IllegalArgumentException if not a member\n\n// after\nif (rbf.membershipTest(fpKey)) {\n  rbf.selectiveClearing(fpKey, RemoveScheme.RATIO);\n} else {\n  LOG.debug(\"skipping {}: not claimed by this filter\", fpKey);\n}","handlingStrategy":"validation","validationCode":"if (rbf.membershipTest(fpKey)) {\n  rbf.selectiveClearing(fpKey, RemoveScheme.RATIO);\n} else {\n  LOG.debug(\"not claimed by this filter; skipping {}\", fpKey);\n}","typeGuard":"static boolean isRetouchable(RetouchedBloomFilter f, Key k) {\n  return k != null && f.membershipTest(k);\n}","tryCatchPattern":"try {\n  rbf.selectiveClearing(k, RemoveScheme.RATIO);\n} catch (IllegalArgumentException e) {\n  LOG.debug(\"key {} is not a member; skipping retouch\", k);\n}","preventionTips":["Always gate selectiveClearing with membershipTest - it is cheap compared to the retouch","Keep the byte encoding used for Keys stable across add and retouch stages","Rebuild false-positive lists from the same filter generation you retouch"],"tags":["hadoop","bloom-filter","java","precondition","state-check"],"backgroundTag":"bloom-filter-membership-check-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}