{"record":{"id":"471a4423a4a20561","repo":"apache/druid","slug":"exception-while-serializing-event","errorCode":null,"errorMessage":"Exception while serializing event","messagePattern":"Exception while serializing event","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"extensions-contrib/kafka-emitter/src/main/java/org/apache/druid/emitter/kafka/KafkaEmitter.java","lineNumber":266,"sourceCode":"        } else if (event instanceof AlertEvent) {\n          if (!eventTypes.contains(EventType.ALERTS) || !alertQueue.offer(objectContainer)) {\n            alertLost.incrementAndGet();\n          }\n        } else if (event instanceof RequestLogEvent) {\n          if (!eventTypes.contains(EventType.REQUESTS) || !requestQueue.offer(objectContainer)) {\n            requestLost.incrementAndGet();\n          }\n        } else if (event instanceof SegmentMetadataEvent) {\n          if (!eventTypes.contains(EventType.SEGMENT_METADATA) || !segmentMetadataQueue.offer(objectContainer)) {\n            segmentMetadataLost.incrementAndGet();\n          }\n        } else {\n          invalidLost.incrementAndGet();\n        }\n      }\n      catch (JsonProcessingException e) {\n        invalidLost.incrementAndGet();\n        log.warn(e, \"Exception while serializing event\");\n      }\n    }\n  }\n\n  private EventMap addExtraDimensionsToEvent(EventMap map)\n  {\n    if (config.getClusterName() != null || config.getExtraDimensions() != null) {\n      EventMap.Builder eventMapBuilder = map.asBuilder();\n      if (config.getClusterName() != null) {\n        eventMapBuilder.put(\"clusterName\", config.getClusterName());\n      }\n      if (config.getExtraDimensions() != null) {\n        eventMapBuilder.putAll(config.getExtraDimensions());\n      }\n      map = eventMapBuilder.build();\n    }\n    return map;\n  }","sourceCodeStart":248,"sourceCodeEnd":284,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-contrib/kafka-emitter/src/main/java/org/apache/druid/emitter/kafka/KafkaEmitter.java#L248-L284","documentation":"KafkaEmitter serializes each Druid event to JSON via Jackson before sending it to a Kafka topic. When Jackson cannot serialize an event (JsonProcessingException), the event is counted as 'invalidLost' and dropped with this warning. The emitter never blocks or retries; the event is permanently lost.","triggerScenarios":"Emitting a Druid event whose content cannot be mapped to JSON — e.g. a Map/bean containing non-serializable objects, self-referential structures, or objects whose getters throw IOException (Jackson's ObjectMapper.writeValueAsString throws JsonProcessingException).","commonSituations":"Custom event/feeder configurations sending unexpected payloads; emitting events containing DateTime or nested objects not configured with the right JavaType/serialization features; version upgrades changing event shape so the mapper's config no longer matches.","solutions":["Inspect the event contents logged by the warning and add or fix Jackson annotations/serializers for the offending type","Ensure the ObjectMapper passed to KafkaEmitter is configured like Druid's default JSON mapper (register JavaTimeModule, disable WRITE_DATES_AS_TIMESTAMPS mismatches, etc.)","Sanitize or filter events before emitting so non-JSON-serializable values are converted to strings","Monitor the invalidLost metric — if it increments, capture a sample event and reproduce serialization in a unit test"],"exampleFix":"// before\nemitter.emit(event); // event contains a non-serializable object\n// after\nMap<String, Object> safeEvent = new HashMap<>(event.toMap()); // or normalize fields to JSON-friendly types\nemitter.emit(new MapEvent(safeEvent));","handlingStrategy":"validation","validationCode":"try {\n  JSON_MAPPER.writeValueAsString(event);\n} catch (JsonProcessingException e) {\n  log.warn(\"Event not serializable, skipping: %s\", e.getMessage());\n  return;\n}","typeGuard":null,"tryCatchPattern":"try { emitter.emit(event); } catch (RuntimeException e) { /* emitter swallows JsonProcessingException; monitor invalidLost metric instead */ }","preventionTips":["Emit only JSON-friendly types (Map, String, Number, Boolean)","Configure the ObjectMapper once, matching Druid's defaults","Unit-test serialization of each custom event type","Monitor the invalidLost counter for jumps"],"tags":["kafka","jackson","json-serialization","emitter"],"backgroundTag":"json-marshal-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}