apache/druid · error · IllegalArgumentException

Failed to deserialize to RoaringBitmap64Counter, input is an

Error message

Failed to deserialize to RoaringBitmap64Counter, input is an invalid base64 string

What it means

decodeStringToByteArray wraps StringUtils.decodeBase64 and converts any IllegalArgumentException from the decoder into IllegalArgumentException("Failed to deserialize to RoaringBitmap64Counter, input is an invalid base64 string"). It is thrown when the String form of a serialized RoaringBitmap64Counter cannot be base64-decoded, meaning the string is malformed (wrong charset, truncated, whitespace/newlines, or not base64 at all).

Source

Thrown at extensions-contrib/druid-exact-count-bitmap/src/main/java/org/apache/druid/query/aggregation/exact/count/bitmap64/Bitmap64ExactCountMergeComplexMetricSerde.java:57

  static RoaringBitmap64Counter deserializeRoaringBitmap64Counter(final Object object)
  {
    if (object instanceof String) {
      return RoaringBitmap64Counter.fromBytes(decodeStringToByteArray((String) object));
    } else if (object instanceof byte[]) {
      return RoaringBitmap64Counter.fromBytes((byte[]) object);
    } else if (object instanceof RoaringBitmap64Counter) {
      return (RoaringBitmap64Counter) object;
    }
    throw new IAE("Cannot deserialize type[%s] to an RoaringBitmap64Counter:", object.getClass().getName());
  }

  private static byte[] decodeStringToByteArray(String string)
  {
    try {
      return StringUtils.decodeBase64(StringUtils.toUtf8(string));
    }
    catch (IllegalArgumentException e) {
      throw new IAE("Failed to deserialize to RoaringBitmap64Counter, input is an invalid base64 string");
    }
  }

  @Override
  public String getTypeName()
  {
    return Bitmap64ExactCountModule.TYPE_NAME; // must be common type name
  }

  @Override
  public ObjectStrategy getObjectStrategy()
  {
    return Bitmap64ExactCountObjectStrategy.STRATEGY;
  }

  @Override
  public ComplexMetricExtractor getExtractor()
  {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Validate the string is well-formed base64 before ingestion: Base64.getDecoder().decode(str) in a try-catch, or a regex check for ^[A-Za-z0-9+/]*={0,2}$.
2. Confirm the producer encodes with base64 (StringUtils.encodeBase64 / Base64 encoder), not hex or raw bytes-as-string.
3. Check the transport: if the string passes through URLs or CSV, ensure '+', '/', and '=' are not mangled (use URL-safe base64 or proper escaping end-to-end).
4. Regenerate the value from the original RoaringBitmap64Counter if the string may have been corrupted or truncated in transit.

Example fix

// before
String s = row.getString(col); // could be hex or invalid
serde.extractValue(s);
// after
String s = row.getString(col);
if (s != null && s.matches("[A-Za-z0-9+/]+={0,2}")) {
  serde.extractValue(s);
} else {
  byte[] raw = Hex.decodeHex(s.toCharArray()); // decode hex, then base64-encode
  s = Base64.getEncoder().encodeToString(raw);
  serde.extractValue(s);
}

static boolean isValidBase64(String s) {
  if (s == null || s.isEmpty()) return false;
  try { java.util.Base64.getDecoder().decode(s); return true; }
  catch (IllegalArgumentException e) { return false; }
}

try {
  RoaringBitmap64Counter c = serde.extractValue(base64String);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("invalid base64")) {
    // re-encode from source bytes or quarantine the record
  } else {
    throw e;
  }
}

When it happens

Trigger: deserializeRoaringBitmap64Counter receives a String whose content is not valid base64 — e.g. an unencoded raw serialization, a hex string, a value with padding/newline issues, or a JSON field that got URL-encoded somewhere upstream.

Common situations: Producing the serialized value with a different bitmap library or encoding (hex instead of base64); manually copying bitmap strings between systems and truncating them; transport layers (HTTP query params, CSV) mangling base64 (e.g. stripping '+'); loading data exported in a non-base64 format.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• unknown event type [%s]
• interrupted flushing elements from queue
• Cannot deserialize type[%s] to an RoaringBitmap64Counter:
• Object cannot be deserialized to a Moments Sketch:
• Object is not of a type that can be deserialized to a quanti

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/e7f5edea9eb24a62. Report an issue: GitHub.