apache/druid · warning

Exception while serializing event

Error message

Exception while serializing event

What it means

KafkaEmitter serializes each Druid event to JSON via Jackson before sending it to a Kafka topic. When Jackson cannot serialize an event (JsonProcessingException), the event is counted as 'invalidLost' and dropped with this warning. The emitter never blocks or retries; the event is permanently lost.

Solutions

  1. Inspect the event contents logged by the warning and add or fix Jackson annotations/serializers for the offending type
  2. Ensure the ObjectMapper passed to KafkaEmitter is configured like Druid's default JSON mapper (register JavaTimeModule, disable WRITE_DATES_AS_TIMESTAMPS mismatches, etc.)
  3. Sanitize or filter events before emitting so non-JSON-serializable values are converted to strings
  4. Monitor the invalidLost metric — if it increments, capture a sample event and reproduce serialization in a unit test

Example fix

// before
emitter.emit(event); // event contains a non-serializable object
// after
Map<String, Object> safeEvent = new HashMap<>(event.toMap()); // or normalize fields to JSON-friendly types
emitter.emit(new MapEvent(safeEvent));
Defensive patterns

Strategy: validation

Validate before calling

try {
  JSON_MAPPER.writeValueAsString(event);
} catch (JsonProcessingException e) {
  log.warn("Event not serializable, skipping: %s", e.getMessage());
  return;
}

Try / catch

try { emitter.emit(event); } catch (RuntimeException e) { /* emitter swallows JsonProcessingException; monitor invalidLost metric instead */ }

Prevention

When it happens

Trigger: Emitting a Druid event whose content cannot be mapped to JSON — e.g. a Map/bean containing non-serializable objects, self-referential structures, or objects whose getters throw IOException (Jackson's ObjectMapper.writeValueAsString throws JsonProcessingException).

Common situations: Custom event/feeder configurations sending unexpected payloads; emitting events containing DateTime or nested objects not configured with the right JavaType/serialization features; version upgrades changing event shape so the mapper's config no longer matches.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/471a4423a4a20561. Report an issue: GitHub.

Appendix: source

Thrown at extensions-contrib/kafka-emitter/src/main/java/org/apache/druid/emitter/kafka/KafkaEmitter.java:266

        } else if (event instanceof AlertEvent) {
          if (!eventTypes.contains(EventType.ALERTS) || !alertQueue.offer(objectContainer)) {
            alertLost.incrementAndGet();
          }
        } else if (event instanceof RequestLogEvent) {
          if (!eventTypes.contains(EventType.REQUESTS) || !requestQueue.offer(objectContainer)) {
            requestLost.incrementAndGet();
          }
        } else if (event instanceof SegmentMetadataEvent) {
          if (!eventTypes.contains(EventType.SEGMENT_METADATA) || !segmentMetadataQueue.offer(objectContainer)) {
            segmentMetadataLost.incrementAndGet();
          }
        } else {
          invalidLost.incrementAndGet();
        }
      }
      catch (JsonProcessingException e) {
        invalidLost.incrementAndGet();
        log.warn(e, "Exception while serializing event");
      }
    }
  }

  private EventMap addExtraDimensionsToEvent(EventMap map)
  {
    if (config.getClusterName() != null || config.getExtraDimensions() != null) {
      EventMap.Builder eventMapBuilder = map.asBuilder();
      if (config.getClusterName() != null) {
        eventMapBuilder.put("clusterName", config.getClusterName());
      }
      if (config.getExtraDimensions() != null) {
        eventMapBuilder.putAll(config.getExtraDimensions());
      }
      map = eventMapBuilder.build();
    }
    return map;
  }

View on GitHub (pinned to 9b90983fd2)