firecracker-microvm/firecracker · critical

vsock: Could not trigger device interrupt

Error message

vsock: Could not trigger device interrupt

What it means

Panic at event_handler.rs:218: after a vsock event was processed in the activated branch, `self.signal_used_queues(&used_queues).expect("vsock: Could not trigger device interrupt")` fires because signaling returned `Err(DeviceError::FailedSignalingIrq)` — the underlying interrupt EventFd write (ioctl WRITE on the irq fd) failed. Unlike errors 60-62, the device WAS activated; the interrupt delivery itself failed.

Source

Thrown at src/vmm/src/devices/virtio/vsock/event_handler.rs:218

        let evset = event.event_set();

        if self.is_activated() {
            let used_queues = match source {
                Self::PROCESS_ACTIVATE => {
                    self.handle_activate_event(ops);
                    Vec::new()
                }
                Self::PROCESS_RXQ => self.handle_rxq_event(evset),
                Self::PROCESS_TXQ => self.handle_txq_event(evset),
                Self::PROCESS_EVQ => self.handle_evq_event(evset),
                Self::PROCESS_NOTIFY_BACKEND => self.notify_backend(evset).unwrap(),
                _ => {
                    warn!("Unexpected vsock event received: {:?}", source);
                    Vec::new()
                }
            };
            self.signal_used_queues(&used_queues)
                .expect("vsock: Could not trigger device interrupt");
        } else {
            warn!(
                "Vsock: The device is not yet activated. Spurious event received: {:?}",
                source
            );
            if source == Self::PROCESS_ACTIVATE {
                let _ = self.activate_evt.read();
            } else {
                self.drain_queue_events();
            }
        }
    }

    fn init(&mut self, ops: &mut EventOps) {
        // This function can be called during different points in the device lifetime:
        //  - shortly after device creation,
        //  - on device activation (is-activated already true at this point),
        //  - on device restore from snapshot.

View on GitHub (pinned to 9384f395f5)

Solutions

1. Check for races between VM shutdown/teardown and vsock event processing — pause or detach the event handler before closing irq fds
2. On restore, ensure the device's interrupt objects are rebuilt with the new irq fds before event handling resumes
3. Replace the `.expect` with proper error propagation so a failed irq write logs and drops the notification instead of aborting the VMM
4. Verify process fd limits (ulimit -n) if eventfd writes fail under load

Example fix

// before
self.signal_used_queues(&used_queues)
    .expect("vsock: Could not trigger device interrupt");

// after
if let Err(e) = self.signal_used_queues(&used_queues) {
    error!("vsock: failed to signal used queues {:?}: {:?}", used_queues, e);
}

// Before dispatching backend events, confirm the irq fd is still alive
// (guard against teardown races by pausing the device event loop first)
if shutting_down { return; } // do not process events during teardown

// The panic escapes the event handler thread; catch at the thread boundary to capture a core/abort cleanly
let result = std::panic::catch_unwind(AssertUnwindSafe(|| handler.handle_event(src, evset)));
if result.is_err() {
    error!("vsock irq signaling failed; stopping device event loop");
    metrics.vsock_irq_failures.add(1);
}

When it happens

Trigger: The device is activated, an RXQ/TXQ/EVQ or backend event is handled, and then `interrupt.trigger_queues(used_queues)` fails — typically EBADF/EFAULT on a closed or invalid irq EventFd (fd torn down during device removal, snapshot restore with stale fd, or fd exhaustion).

Common situations: Snapshot/restore where irq fds were recreated but the device still holds the old ones; racing device teardown (VM stop) with an in-flight backend event; exceeding the process fd limit so the eventfd write fails.

Related errors

• Device is not initialized
• GDB requires KVM
• Error getting initial gdb event
• Error converting cpu id to Tid
• GDB requires KVM

AI-assisted analysis of firecracker-microvm/firecracker@9384f395f5 (2026-08-19). Data as JSON: /api/errors/a3ab306b291eb664. Report an issue: GitHub.