What causes "Undefined selective clearing scheme" in apache/hadoop?

Always pass the named constants RemoveScheme.RANDOM / MINIMUM_FN / MAXIMUM_FP / RATIO, never raw shorts

apache/hadoop · error · AssertionError

Undefined selective clearing scheme

Error message

Undefined selective clearing scheme

What it means

selectiveClearing switches over the four RemoveScheme constants RetouchedBloomFilter implements: RANDOM=0, MINIMUM_FN=1, MAXIMUM_FP=2, RATIO=3. Any other short hits the default branch and throws AssertionError, i.e. a broken invariant caused by the caller passing an out-of-range scheme value.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/RetouchedBloomFilter.java:230

    case RANDOM:
      index = randomRemove();
      break;
    
    case MINIMUM_FN:
      index = minimumFnRemove(h);
      break;
    
    case MAXIMUM_FP:
      index = maximumFpRemove(h);
      break;
    
    case RATIO:
      index = ratioRemove(h);
      break;
    
    default:
      throw new AssertionError("Undefined selective clearing scheme");

    }

    clearBit(index);
  }

  private int randomRemove() {
    if (rand == null) {
      rand = new Random();
    }

    return rand.nextInt(nbHash);
  }

  /**
   * Chooses the bit position that minimizes the number of false negative generated.
   * @param h The different bit positions.
   * @return The position that minimizes the number of false negative generated.

View on GitHub (pinned to 2add963021)

Solutions

Always pass the named constants RemoveScheme.RANDOM / MINIMUM_FN / MAXIMUM_FP / RATIO, never raw shorts
Validate externally sourced scheme values against 0..3 (or parse into an enum) before use
Bounds-check scheme values right after deserialization

Example fix

// before
rbf.selectiveClearing(k, schemeFromFile); // raw short: (short) 4 -> AssertionError

// after
if (schemeFromFile < RemoveScheme.RANDOM || schemeFromFile > RemoveScheme.RATIO) {
  throw new IOException("invalid remove scheme: " + schemeFromFile);
}
rbf.selectiveClearing(k, schemeFromFile);

Defensive patterns

Strategy: validation

Validate before calling

if (scheme < RemoveScheme.RANDOM || scheme > RemoveScheme.RATIO) {
  throw new IOException("invalid remove scheme: " + scheme);
}
rbf.selectiveClearing(k, scheme);

Type guard

static boolean isValidRemoveScheme(short scheme) {
  return scheme >= RemoveScheme.RANDOM && scheme <= RemoveScheme.RATIO;
}

Prevention

Never pass raw shorts - always the RemoveScheme constants
Parse external scheme values into an enum at the boundary and reject unknowns there
Do not catch AssertionError in production code; fix the caller instead

When it happens

Trigger: selectiveClearing(k, (short) 4) or a negative value; scheme IDs read from a file/protocol that defines more schemes than this class supports; hand-mapping an enum to raw shorts and missing a case.

Common situations: Config-driven scheme selection; version skew where a producer emits a scheme this build lacks; copy-paste of magic numbers instead of the named constants.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/065aea8b42410d39. Report an issue: GitHub.