prometheus/node_exporter · error

failed to scan DM-multipath devices

Error message

failed to scan DM-multipath devices: %w

What it means

Update calls fs.DMMultipathDevices() to scan /sys/block/dm-* for multipath devices. ErrNotExist and ErrPermission are demoted to debug logs plus ErrNoData, but any other error aborts the scrape with this wrapped message. It signals an unexpected failure enumerating device-mapper block devices rather than their simple absence.

Solutions

1. Check kernel logs (dmesg) for device-mapper or sysfs I/O errors at scrape time
2. Inspect the wrapped cause (%w) to identify the failing device or errno
3. Re-scan; transient races during dm device teardown usually clear on the next scrape
4. Verify /sys/block is healthy and dm-* entries are not stale (dmsetup ls)

if _, err := os.Stat("/sys/block"); err != nil {
    // /sys/block unreadable; expect ErrNoData from dm-multipath collector
}

if err := c.Update(ch); err != nil {
    if errors.Is(err, collector.ErrNoData) {
        // tolerated: dm devices absent or unreadable
    } else if strings.Contains(err.Error(), "failed to scan DM-multipath devices") {
        // inspect wrapped cause; often transient during device teardown — retry next scrape
    }
}

When it happens

Trigger: DMMultipathDevices() returns a non-ErrNotExist/ErrPermission error during a scrape: I/O error reading /sys/block, glob failure, or unexpected filesystem error while reading dm-* attributes (name, uuid).

Common situations: Failing underlying storage while dm devices are being removed/reconfigured mid-scan; sysfs returning EIO for a stale dm device; exotic LSM/SELinux denials reported as other errno values.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• failed to retrieve bcache stats
• failed to retrieve bcachefs stats
• failed to retrieve Btrfs stats from procfs
• failed to open sysfs
• couldn't get NUMA meminfo

AI-assisted analysis of prometheus/node_exporter@17ddd77c59 (2026-09-07). Data as JSON: /api/errors/b95c0d848ac6ffa4. Report an issue: GitHub.

Appendix: source

Thrown at collector/dmmultipath_linux.go:104

	fs, err := blockdevice.NewFS(*procPath, *sysPath)
	if err != nil {
		return nil, fmt.Errorf("failed to open sysfs: %w", err)
	}

	return &dmMultipathCollector{
		fs:     fs,
		logger: logger,
	}, nil
}

func (c *dmMultipathCollector) Update(ch chan<- prometheus.Metric) error {
	devices, err := c.fs.DMMultipathDevices()
	if err != nil {
		if errors.Is(err, os.ErrNotExist) || errors.Is(err, os.ErrPermission) {
			c.logger.Debug("Could not read DM-multipath devices", "err", err)
			return ErrNoData
		}
		return fmt.Errorf("failed to scan DM-multipath devices: %w", err)
	}

	for _, dev := range devices {
		ch <- prometheus.MustNewConstMetric(dmmultipathDeviceInfo, prometheus.GaugeValue, 1,
			dev.Name, dev.SysfsName, dev.UUID)

		active := 0.0
		if !dev.Suspended {
			active = 1.0
		}
		ch <- prometheus.MustNewConstMetric(dmmultipathDeviceActive, prometheus.GaugeValue, active, dev.Name, dev.SysfsName)
		ch <- prometheus.MustNewConstMetric(dmmultipathDeviceSizeBytes, prometheus.GaugeValue, float64(dev.SizeBytes), dev.Name, dev.SysfsName)

		var activePaths, failedPaths float64
		for _, p := range dev.Paths {
			if isPathActive(p.State) {
				activePaths++
			} else {

View on GitHub (pinned to 17ddd77c59)