{"record":{"id":"97d304324de078e9","repo":"hashicorp/nomad","slug":"errpoolexhausted","errorCode":"ErrPoolExhausted","errorMessage":"users: uid/gid pool exhausted","messagePattern":"users: uid/gid pool exhausted","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"helper/users/dynamic/pool.go","lineNumber":19,"sourceCode":"// Copyright IBM Corp. 2015, 2026\n// SPDX-License-Identifier: BUSL-1.1\n\n// Package dynamic provides a way of allocating UID/GID to be used by Nomad\n// tasks with no associated service users managed by the operating system.\npackage dynamic\n\nimport (\n\t\"errors\"\n\t\"math/rand\"\n\t\"strconv\"\n\t\"sync\"\n\n\t\"github.com/hashicorp/go-set/v3\"\n\t\"github.com/hashicorp/nomad/helper\"\n)\n\nvar (\n\tErrPoolExhausted = errors.New(\"users: uid/gid pool exhausted\")\n\tErrReleaseUnused = errors.New(\"users: release of unused uid/gid\")\n\tErrCannotParse   = errors.New(\"users: unable to parse uid/gid from username\")\n)\n\n// none indicates no dynamic user\nconst none = 0\n\n// doNotEnable indicates functionality should be disabled\nconst doNotEnable = -1\n\n// A UGID is a combination User (UID) and Group (GID). Since Nomad is\n// allocating these values together from the same pool it can ensure they are\n// always matching values, thus encoding them with one value.\ntype UGID int\n\n// String returns the string representation of a UGID.\n//\n// It's just the numbers.","sourceCodeStart":1,"sourceCodeEnd":37,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/users/dynamic/pool.go#L1-L37","documentation":"ErrPoolExhausted is returned by the dynamic uid/gid pool's Acquire when every ugID in the configured min..max range is already in use. The pool tracks allocated users in a set; when used.Size() equals the full range size, no more dynamic users can be handed out to tasks.","triggerScenarios":"Calling Pool.Acquire() when the number of concurrently held dynamic users equals (p.max - p.min) + 1; i.e. all IDs in the configured range are allocated and none have been released.","commonSituations":"Nomad clusters running many tasks with dynamic workload users enabled where the configured uid/gid range is too small for concurrent task count; leaked allocations that never released their ugids after crashes or client restarts.","solutions":["Increase the dynamic user uid/gid min/max range in client configuration to cover peak concurrent tasks","Ensure tasks/allocations release their acquired ugids (Pool.Release) on shutdown","Investigate leaked allocations holding ugids and restart the Nomad client to reset the pool","Reduce per-task dynamic user usage so fewer ugids are held concurrently"],"exampleFix":"// before: pool range too small for 100 concurrent tasks\npool, _ := dynamic.NewPool(5000, 5100) // only 101 ids\n// after: range sized to workload\npool, _ := dynamic.NewPool(5000, 9000) // 4001 ids","handlingStrategy":"validation","validationCode":"if pool.Size() >= int(max-min)+1 {\n  return errors.New(\"uid/gid pool would be exhausted; increase range or release ugids\")\n}","typeGuard":null,"tryCatchPattern":"ugid, err := p.Acquire()\nif errors.Is(err, dynamic.ErrPoolExhausted) {\n  // back off / shed load / resize pool range\n  return retryLater()\n}","preventionTips":["Size the min/max ugID range to exceed peak concurrent task count","Always Release acquired ugids on allocation shutdown, including error paths","Monitor pool usage (used count vs capacity) in client metrics","Restart clients to clear leaks if exhaustion persists without matching task count"],"tags":["go","resource-exhaustion","nomad","pool"],"backgroundTag":"resource-pool-exhausted","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}