{"record":{"id":"706d7cc52fb9ee86","repo":"prometheus/node_exporter","slug":"unable-to-retrieve-number-of-allocated-threads-w","errorCode":null,"errorMessage":"unable to retrieve number of allocated threads: %w","messagePattern":"unable to retrieve number of allocated threads: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"collector/processes_linux.go","lineNumber":88,"sourceCode":"\t\t),\n\t\tprocsState: prometheus.NewDesc(\n\t\t\tprometheus.BuildFQName(namespace, subsystem, \"state\"),\n\t\t\t\"Number of processes in each state.\",\n\t\t\t[]string{\"state\"}, nil,\n\t\t),\n\t\tpidUsed: prometheus.NewDesc(prometheus.BuildFQName(namespace, subsystem, \"pids\"),\n\t\t\t\"Number of PIDs\", nil, nil,\n\t\t),\n\t\tpidMax: prometheus.NewDesc(prometheus.BuildFQName(namespace, subsystem, \"max_processes\"),\n\t\t\t\"Number of max PIDs limit\", nil, nil,\n\t\t),\n\t\tlogger: logger,\n\t}, nil\n}\nfunc (c *processCollector) Update(ch chan<- prometheus.Metric) error {\n\tpids, states, threads, threadStates, err := c.getAllocatedThreads()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"unable to retrieve number of allocated threads: %w\", err)\n\t}\n\n\tch <- prometheus.MustNewConstMetric(c.threadAlloc, prometheus.GaugeValue, float64(threads))\n\tmaxThreads, err := readUintFromFile(procFilePath(\"sys/kernel/threads-max\"))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"unable to retrieve limit number of threads: %w\", err)\n\t}\n\tch <- prometheus.MustNewConstMetric(c.threadLimit, prometheus.GaugeValue, float64(maxThreads))\n\n\tfor state := range states {\n\t\tch <- prometheus.MustNewConstMetric(c.procsState, prometheus.GaugeValue, float64(states[state]), state)\n\t}\n\n\tfor state := range threadStates {\n\t\tch <- prometheus.MustNewConstMetric(c.threadsState, prometheus.GaugeValue, float64(threadStates[state]), state)\n\t}\n\n\tpidM, err := readUintFromFile(procFilePath(\"sys/kernel/pid_max\"))","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/prometheus/node_exporter/blob/17ddd77c59ba27e1508e9f7894b1e55b44d6aed3/collector/processes_linux.go#L70-L106","documentation":"processCollector.Update wraps errors from getAllocatedThreads with this message. getAllocatedThreads enumerates all processes via procfs (fs.AllProcs()) and reads each /proc/<pid>/stat; any failure listing processes or reading stats surfaces here as 'unable to retrieve number of allocated threads'.","triggerScenarios":"Update() is called on a scrape and c.getAllocatedThreads() returns an error: fs.AllProcs() failed (145), a pid's stat read failed non-fatally (146), or getThreadStates failed (147-149).","commonSituations":"Scraping on a system where /proc/<pid> entries vanish rapidly (short-lived processes under heavy churn); restricted permissions on /proc; hidepid=2 mount option hiding other users' processes from the exporter user.","solutions":["Check the wrapped cause (%w) in logs: if it's 'unable to list all processes' or 'error reading stat for pid', inspect /proc access for the exporter user.","If hidepid is in effect, run the exporter as root or grant it the required group (hidepid=gid=...) so it can read all /proc/<pid>/stat entries.","Confirm the failing read is transient (pid race) — node_exporter already ignores ENOENT-style errors via isIgnoredError; persistent failures indicate a permissions or procfs issue.","Re-run the scrape; a single failed scrape with this error during process churn is usually harmless."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// before scraping\nif _, err := os.Stat(\"/proc/self/stat\"); err != nil {\n    log.Printf(\"procfs unreadable for exporter user: %v\", err)\n}","typeGuard":null,"tryCatchPattern":"if err := c.Update(ch); err != nil {\n    var transient bool\n    if strings.Contains(err.Error(), \"no such file\") {\n        transient = true // pid race: safe to retry next scrape\n    }\n    log.Printf(\"processes update failed (transient=%v): %v\", transient, err)\n}","preventionTips":["Run the exporter as a user that can read all /proc/<pid> entries (root, or hidepid gid membership).","Expect occasional transient failures during extreme pid churn; alert on sustained failures only.","Keep node_exporter/procfs up to date for broader ignored-error handling.","Monitor scrape_duration and failed scrapes to distinguish races from systemic permission issues."],"tags":["linux","procfs","processes","node-exporter","scrape"],"backgroundTag":"file-read-failed","analyzedSha":"17ddd77c59ba27e1508e9f7894b1e55b44d6aed3","analyzedAt":"2026-09-07T17:54:06.211Z","contentChangedAt":"2026-09-07T17:54:06.211Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}