{"record":{"id":"f674f1b4ca195056","repo":"stalwartlabs/stalwart","slug":"missing-span-id","errorCode":null,"errorMessage":"Missing span ID","messagePattern":"Missing span ID","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/trc/src/ipc/collector.rs","lineNumber":164,"sourceCode":"\n                                    if self.active_spans.len() > STALE_SPAN_CHECK_WATERMARK {\n                                        self.active_spans.retain(|_, span| {\n                                            timestamp.saturating_sub(span.inner.timestamp)\n                                                < SPAN_MAX_HOLD\n                                        });\n                                    }\n                                    event\n                                }\n\n                                HTTP_CONN_END\n                                | IMAP_CONN_END\n                                | POP3_CONN_END\n                                | SMTP_CONN_END\n                                | MANAGE_SIEVE_CONN_END\n                                | EV_ATTEMPT_END => {\n                                    if let Some(span) = self\n                                        .active_spans\n                                        .remove(&event.span_id().expect(\"Missing span ID\"))\n                                    {\n                                        event.inner.span = Some(span.clone());\n                                    } else {\n                                        #[cfg(any(feature = \"dev_mode\", feature = \"test_mode\"))]\n                                        {\n                                            if event.span_id().unwrap() != 0 {\n                                                eprintln!(\"Unregistered span ID: {event:?}\");\n                                            }\n                                        }\n                                    }\n                                    Arc::new(event)\n                                }\n                                _ => {\n                                    if let Some(span_id) = event.span_id() {\n                                        if let Some(span) = self.active_spans.get(&span_id) {\n                                            event.inner.span = Some(span.clone());\n                                        } else {\n                                            #[cfg(any(","sourceCodeStart":146,"sourceCodeEnd":182,"githubUrl":"https://github.com/stalwartlabs/stalwart/blob/e96200385781a6a9995a8b839ac27d6c75a983ee/crates/trc/src/ipc/collector.rs#L146-L182","documentation":"When the collector processes a connection-end event (IMAP/POP3/SMTP/ManageSieve connection end or attempt end) it removes the corresponding active span keyed by the event's span ID. The event carries no span ID (span_id() returned None), so the expect panics. This indicates an event that should always carry a span ID arrived without one — e.g. an end event emitted without a matching start.","triggerScenarios":"Receiving *_CONN_END or EV_ATTEMPT_END telemetry events whose span_id() is None, i.e. events emitted outside an instrumented span or produced by a mismatched event instrumentation.","commonSituations":"A connection was closed after server restart (start event lost), mixed server versions writing incompatible event formats to the shared event stream, or a code path emitting end events without entering the span.","solutions":["Upgrade all nodes writing events to the same version so span IDs are always emitted with end events","Make the code tolerate missing span IDs (use match/if-let with a default of 0 instead of expect)","Clear stale event queues/spans after a restart to avoid orphan end events","Audit instrumentation so every *_CONN_END emission is paired with a started span"],"exampleFix":"// before\nif let Some(span) = self.active_spans.remove(&event.span_id().expect(\"Missing span ID\")) {\n// after\nif let Some(span_id) = event.span_id() {\n    if let Some(span) = self.active_spans.remove(&span_id) {","handlingStrategy":"type-guard","validationCode":"if event.span_id().is_none() { log::warn!(\"end event without span id, skipping\"); return; }","typeGuard":"fn has_span_id(e: &Event) -> bool { e.span_id().is_some() }","tryCatchPattern":"if let Some(id) = event.span_id() {\n    if let Some(span) = self.active_spans.remove(&id) { event.inner.span = Some(span.clone()); }\n}","preventionTips":["Always emit end events inside the span they terminate","Keep server versions homogeneous when sharing event streams","Handle span_id() with Option semantics instead of expect/unwrap","Reset active span state on restart to drop orphaned end events"],"tags":["telemetry","tracing","collector","panic"],"backgroundTag":"null-argument","analyzedSha":"e96200385781a6a9995a8b839ac27d6c75a983ee","analyzedAt":"2026-09-06T22:07:17.982Z","contentChangedAt":"2026-09-06T22:07:17.982Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}