prometheus/node_exporter · error

couldn't get diskstats

Error message

couldn't get diskstats: %w

What it means

The Linux diskstats collector's Update reads per-device I/O statistics from /proc/diskstats via c.fs.ProcDiskstats(). When that read/parse fails, Update returns "couldn't get diskstats: %w" and the scrape reports an error for that collector.

Solutions

1. Check readability of /proc/diskstats as the node_exporter user: cat /proc/diskstats.
2. In containers, ensure /proc is mounted (e.g. -v /proc:/host/proc:ro with --path.procfs=/host/proc).
3. Check hidepid procfs mount options and LSM/audit logs for denials; grant read access to the exporter user.
4. Inspect the wrapped cause (%w); if parsing fails with a custom kernel, update node_exporter/procfs or file an upstream issue.
5. Disable diskstats collection if not needed (--no-collector.diskstats).

Example fix

// before (hidepid mount blocks read)
proc /proc proc rw,hidepid=2 0 0
// after (allow exporter group)
proc /proc proc rw,hidepid=2,gid=<exporter_gid> 0 0

f, err := os.Open("/proc/diskstats")
if err != nil {
    // /proc/diskstats unreadable; diskstats collector will fail
} else {
    f.Close()
}

if err := coll.Update(ch); err != nil {
    var perr *fs.PathError
    if errors.As(err, &perr) && errors.Is(perr, fs.ErrPermission) {
        log.Printf("diskstats blocked by permissions: %v", perr)
        return nil
    }
    return err
}

When it happens

Trigger: Update() on Linux when ProcDiskstats() errors: /proc/diskstats missing or unreadable for the exporter process, or unexpected content the blockdevice parser cannot handle (very unusual kernel output).

Common situations: Containers without /proc mounted or with restrictive procfs hidepid settings; LSM/SELinux policies denying read of /proc/diskstats; heavily customized kernels producing unexpected diskstats formatting; chrooted environments lacking procfs.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• failed to get vulnerabilities
• failed to open sysfs
• failed to retrieve nfs stats
• failed to retrieve nfsd stats
• interrupts empty

AI-assisted analysis of prometheus/node_exporter@17ddd77c59 (2026-09-07). Data as JSON: /api/errors/51496d4d37d06a44. Report an issue: GitHub.

Appendix: source

Thrown at collector/diskstats_linux.go:266

			},
		},
		logger: logger,
	}

	// Only enable getting device properties from udev if the directory is readable.
	if stat, err := os.Stat(*udevDataPath); err != nil || !stat.IsDir() {
		logger.Error("Failed to open directory, disabling udev device properties", "path", *udevDataPath)
	} else {
		collector.getUdevDeviceProperties = getUdevDeviceProperties
	}

	return &collector, nil
}

func (c *diskstatsCollector) Update(ch chan<- prometheus.Metric) error {
	diskStats, err := c.fs.ProcDiskstats()
	if err != nil {
		return fmt.Errorf("couldn't get diskstats: %w", err)
	}

	for _, stats := range diskStats {
		dev := stats.DeviceName
		if c.deviceFilter.ignored(dev) {
			continue
		}

		// Only fetch udev device properties when udev is available
		// to avoid unnecessary file I/O.
		var info udevInfo
		if c.getUdevDeviceProperties != nil {
			var err error
			info, err = c.getUdevDeviceProperties(stats.MajorNumber, stats.MinorNumber)
			if err != nil {
				c.logger.Debug("Failed to parse udev info", "err", err)
			}
		}

View on GitHub (pinned to 17ddd77c59)