What causes "couldn't get kvm: %w" in prometheus/node_exporter?

Ensure node_exporter runs with permission to open kvm (usually root, or group access to /dev/mem)

couldn't get kvm

Error message

couldn't get kvm: %w

What it means

After emitting sysctl-based memory metrics, Update uses the kvm library (c.kvm.SwapUsedPages()) to read swap usage from the kernel swap device list. If that kvm read fails, the scrape aborts with this wrapped error. It indicates node_exporter could not access kernel memory/swap information via /dev/kvm (kvm_getswapinfo).

Solutions

Ensure node_exporter runs with permission to open kvm (usually root, or group access to /dev/mem)
Verify /dev/mem and swap device nodes exist inside the environment (mount devfs in jails)
Rebuild/redeploy node_exporter matching the running kernel version to fix kvm symbol mismatch
As a workaround, disable/monitor swap metrics or run the exporter outside the restricted jail

Example fix

// before
swapUsed, err := c.kvm.SwapUsedPages()
if err != nil {
	return fmt.Errorf("couldn't get kvm: %w", err)
}
// after (deployment fix, run exporter with kvm access, e.g. in rc.d/Monitoring)
// pkg add node_exporter && service node_exporter enable  # runs as root by default
// or grant: pw groupmod kvm -m exporter-user  (if using a kvm group setup)

Defensive patterns

Strategy: try-catch

Validate before calling

// deployment-time check: kvm_openfiles must succeed
// test -r /dev/mem && echo 'kvm access ok' || echo 'grant /dev/mem access'

Try / catch

if err := c.Update(ch); err != nil {
	if strings.Contains(err.Error(), "couldn't get kvm") {
		logger.Warn("swap metrics unavailable (kvm access failed)", "err", err)
		return nil // drop swap metrics, keep memory metrics
	}
	return err
}

Prevention

Run node_exporter as root or with /dev/mem read access
Keep the exporter binary in sync with kernel version (kvm symbol tables)
Mount devfs with mem device in jails

When it happens

Trigger: A scrape when kvm_openfiles/kvm_getswapinfo fails — typically /dev/mem or /dev/kdevfs access problems, wrong kvm symbol tables after a kernel upgrade, or running without sufficient privileges.

Common situations: Running node_exporter in a jail without /dev/mem access; kernel world mismatch (userland kvm lib can't read the running kernel); permission denied on kvm device nodes.

Related errors

AI-assisted analysis of prometheus/node_exporter@17ddd77c59 (2026-09-07). Data as JSON: /api/errors/8bf46b68646a96b0. Report an issue: GitHub.

Appendix: source

Thrown at collector/memory_bsd.go:169

			return fmt.Errorf("couldn't get memory: %w", err)
		}

		// Most are gauges.
		if m.valueType == 0 {
			m.valueType = prometheus.GaugeValue
		}

		ch <- prometheus.MustNewConstMetric(
			prometheus.NewDesc(
				prometheus.BuildFQName(namespace, memorySubsystem, m.name),
				m.description,
				nil, nil,
			), m.valueType, v)
	}

	swapUsed, err := c.kvm.SwapUsedPages()
	if err != nil {
		return fmt.Errorf("couldn't get kvm: %w", err)
	}

	ch <- prometheus.MustNewConstMetric(
		prometheus.NewDesc(
			prometheus.BuildFQName(namespace, memorySubsystem, "swap_used_bytes"),
			"Currently allocated swap",
			nil, nil,
		), prometheus.GaugeValue, float64(swapUsed*c.pageSize))

	return nil
}

View on GitHub (pinned to 17ddd77c59)