apache/druid · error · IllegalStateException

Got an exception while parsing file [%s]

Error message

Got an exception while parsing file [%s]

What it means

mergeBloomFilterByteBuffers computes each buffer's serialized length as START_OF_SERIALIZED_LONGS + (declared bitset longs * Long.BYTES) and requires the two lengths to be equal before merging. If they differ, it throws this IllegalArgumentException. The serialized length encodes numBits, so this check rejects bloom filters of different sizes.

Source

Thrown at extensions-contrib/ambari-metrics-emitter/src/main/java/org/apache/druid/emitter/ambari/metrics/WhiteListBasedDruidToTimelineEventConverter.java:225

  private ImmutableSortedMap<String, ImmutableList<String>> readMap(final String mapPath)
  {
    String fileContent;
    String actualPath = mapPath;
    try {
      if (Strings.isNullOrEmpty(mapPath)) {
        URL defaultWhiteListMapUrl = this.getClass().getClassLoader().getResource("defaultWhiteListMap.json");
        actualPath = defaultWhiteListMapUrl.getFile();
        LOGGER.info("using default whiteList map located at [%s]", actualPath);
        fileContent = Resources.toString(defaultWhiteListMapUrl, StandardCharsets.UTF_8);
      } else {
        fileContent = Files.asCharSource(new File(mapPath), StandardCharsets.UTF_8).read();
      }
      return mapper.readerFor(new TypeReference<ImmutableSortedMap<String, ImmutableList<String>>>()
      {
      }).readValue(fileContent);
    }
    catch (IOException e) {
      throw new ISE(e, "Got an exception while parsing file [%s]", actualPath);
    }
  }

  @Override
  public boolean equals(Object o)
  {
    if (this == o) {
      return true;
    }
    if (o == null || getClass() != o.getClass()) {
      return false;
    }

    WhiteListBasedDruidToTimelineEventConverter that = (WhiteListBasedDruidToTimelineEventConverter) o;

    if (namespacePrefix != null ? !namespacePrefix.equals(that.namespacePrefix) : that.namespacePrefix != null) {
      return false;
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Use filters produced with the same bloomFilterNumBits (and numHashFunctions) everywhere before merging.
2. Check START_OF_SERIALIZED_LONGS + (getInt(1+start) * Long.BYTES) for both buffers yourself and route mismatched filters to separate merges.
3. Ensure start offsets (bf1Start/bf2Start) actually point at filter headers, not mid-buffer, so the computed lengths are the real ones.
4. Catch IllegalArgumentException and report which of the two buffers has the wrong size.

Example fix

// before
BloomKFilter.mergeBloomFilterByteBuffers(buf1, buf2, 0, 0); // throws if sizes differ

// after
int len1 = BloomKFilter.START_OF_SERIALIZED_LONGS + (buf1.getInt(1) * Long.BYTES);
int len2 = BloomKFilter.START_OF_SERIALIZED_LONGS + (buf2.getInt(1) * Long.BYTES);
if (len1 != len2) {
  throw new IllegalStateException("cannot merge bloom filters of different sizes: " + len1 + " vs " + len2);
}
BloomKFilter.mergeBloomFilterByteBuffers(buf1, buf2, 0, 0);

int serializedLen(ByteBuffer buf, int start) {
  int longs = buf.duplicate().order(ByteOrder.BIG_ENDIAN).getInt(1 + start);
  return BloomKFilter.START_OF_SERIALIZED_LONGS + longs * Long.BYTES;
}
// call: assert serializedLen(bf1Buffer, bf1Start) == serializedLen(bf2Buffer, bf2Start);

boolean sameSize(ByteBuffer b1, int s1, ByteBuffer b2, int s2) {
  return serializedLen(b1, s1) == serializedLen(b2, s2);
}

try {
  BloomKFilter.mergeBloomFilterByteBuffers(bf1Buffer, bf2Buffer, start1, start2);
} catch (IllegalArgumentException e) {
  throw new IllegalStateException("Refusing to merge bloom filters of different sizes; align bloomFilterNumBits across stages", e);
}

When it happens

Trigger: Calling BloomKFilter.mergeBloomFilterByteBuffers(bf1Buffer, bf2Buffer, start1, start2) with buffers whose declared bitset lengths (bytes at offset 1+start) differ, e.g. filters built with different numBits parameters.

Common situations: Merging bloom filter results from query stages that used different bloomFilterNumBits settings; merging base and offset buffers pointing at filters of different sizes; combining filters serialized by different configurations or external tools.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Emit called unexpectedly before service start
• Unable to copy key [%s] to file [%s]
• Cannot create bloom filter %s for invalid column type [%s]
• unknown event type [%s]
• interrupted flushing elements from queue

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/7375d7cd18069d46. Report an issue: GitHub.