{"record":{"id":"4c373cb1dcd201b4","repo":"stamparm/maltrail","slug":"unhandled-panic-in-process-packet-worker","errorCode":null,"errorMessage":"unhandled panic in process_packet (worker {})","messagePattern":"unhandled panic in process_packet \\(worker (.+?)\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sensor/src/worker.rs","lineNumber":183,"sourceCode":"                    // distorts the thing it reports. Sampling costs <1 ns/packet and the estimate\n                    // is within a couple of percent on any realistic packet mix.\n                    timing_countdown -= 1;\n                    let started = if timing_countdown == 0 {\n                        timing_countdown = TIMING_SAMPLE_STRIDE;\n                        Some(Instant::now())\n                    } else {\n                        None\n                    };\n                    let ip_offset = st.dlt.resolve(datalink, data);\n                    if let Some(offset) = ip_offset {\n                        // Mirrors sensor.py's blanket `except Exception` around\n                        // _process_packet: a parser bug must never take the sensor down.\n                        let result = catch_unwind(AssertUnwindSafe(|| {\n                            process::process_packet(&mut st, data, sec, usec, offset);\n                        }));\n                        if result.is_err() {\n                            st.metrics.panics_recovered += 1;\n                            crate::output::log_error(\n                                &format!(\"unhandled panic in process_packet (worker {})\", ctx.id),\n                                true,\n                            );\n                        }\n                    } else {\n                        st.metrics.packets_ignored += 1;\n                    }\n                    if let Some(started) = started {\n                        st.metrics.processing_nanos += started.elapsed().as_nanos() as u64;\n                        st.metrics.processing_samples += 1;\n                    }\n\n                    if drained >= HOUSEKEEPING_INTERVAL as u32 {\n                        break;\n                    }\n                }\n                // offline: end of file. live: nothing available right now.\n                Ok(None) => {","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/stamparm/maltrail/blob/77cfb06d7606506d101bbcec0786c77166c4255e/sensor/src/worker.rs#L165-L201","documentation":"A worker thread's process_packet call panicked; the sensor catches the unwind so a parser bug cannot take down the whole sensor, increments panics_recovered, and logs this error naming the worker. The offending packet is dropped but processing continues on the next packet.","triggerScenarios":"run_all's per-packet catch_unwind(AssertUnwindSafe(|| process::process_packet(&mut st, data, sec, usec, offset))) returns Err — process_packet panicked on a particular packet's bytes, timestamp, or offset value.","commonSituations":"Crafted or malformed packet data triggering an unwrap/overflow/index bug in a protocol parser; offset from the DLT heuristic (see the unexpected-datalink path) pointing mid-header; a new protocol variant in live traffic hitting an untested parser branch.","solutions":["Capture the offending packet: enable packet dump or note the timestamp (sec, usec) so the triggering bytes can be reproduced in a test.","Report/reproduce via a minimized pcap — panics in process_packet are parser bugs and should be filed with the triggering packet bytes and offset.","Verify panics_recovered in metrics to scope the impact (how many packets are being dropped).","Update the sensor to a version with the parser fix; as an interim, exclude the offending traffic source or interface if the panic rate is high."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// the sensor already isolates the panic per-packet; operators should\n// watch the recovery counter and file a bug with the triggering packet\nif st.metrics.panics_recovered_delta > 0 {\n    dump_ring_buffer_packet(sec, usec, offset); // capture bytes for reproduction\n    file_parser_bug(packet_bytes, offset);\n}","preventionTips":["Add fuzz tests for protocol parsers using real-world pcaps","Avoid unwrap/indexing without bounds checks in process_packet parsers","Track panics_recovered in metrics; any increase indicates a parser bug","Reproduce panics offline with a minimized pcap and fix before upgrading traffic volumes"],"tags":["panic","parser","packet-processing","worker"],"backgroundTag":"internal-invariant-violation","analyzedSha":"77cfb06d7606506d101bbcec0786c77166c4255e","analyzedAt":"2026-09-13T03:50:16.010Z","contentChangedAt":"2026-09-13T03:50:16.010Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}