{"record":{"id":"cef1c10c8d9cbdc4","repo":"hashicorp/nomad","slug":"failed-to-set-subtree-control-on-cpuset-reserve-pa","errorCode":null,"errorMessage":"failed to set subtree control on cpuset reserve partition: %w","messagePattern":"failed to set subtree control on cpuset reserve partition: %w","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/lib/cgroupslib/init.go","lineNumber":186,"sourceCode":"\t\t\treturn fmt.Errorf(\"failed to create share cgroup: %w\", err)\n\t\t}\n\n\t\tif err := writeCG(activation, NomadCgroupParent, SharePartition(), subtreeFile); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to set subtree control on cpuset share partition: %w\", err)\n\t\t}\n\n\t\tlog.Debug(\"partition member nomad.slice/share cgroup initialized\")\n\n\t\t//\n\t\t// configuring nomad.slice/reserve (member)\n\t\t//\n\n\t\tif err := mkCG(NomadCgroupParent, ReservePartition()); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to create share cgroup: %w\", err)\n\t\t}\n\n\t\tif err := writeCG(activation, NomadCgroupParent, ReservePartition(), subtreeFile); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to set subtree control on cpuset reserve partition: %w\", err)\n\t\t}\n\n\t\tlog.Debug(\"partition member nomad.slice/reserve cgroup initialized\")\n\t}\n\n\treturn nil\n}\n\n// detectMemsCG1 will determine the cpuset.mems value to use for\n// Nomad managed cgroups.\n//\n// Copy the value from the root cgroup cpuset.mems file, unless the nomad\n// parent cgroup exists with a value set, in which case use the cpuset.mems\n// value from there.\nfunc detectMemsCG1() (string, error) {\n\t// read root cgroup mems file\n\tmemsRootPath := filepath.Join(root, \"cpuset\", memsFile)\n\tb, err := os.ReadFile(memsRootPath)","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/lib/cgroupslib/init.go#L168-L204","documentation":"After creating nomad.slice/reserve, Init() writes the controller activation list to its cgroup.subtree_control so per-task cgroups beneath reserve can use the controllers. Failure leaves the reserve partition unable to host task cgroups.","triggerScenarios":"writeCG(activation, NomadCgroupParent, ReservePartition(), subtreeFile) fails — EACCES writing the file, EBUSY because processes already reside in nomad.slice/reserve, or the controllers to activate are unavailable on this kernel.","commonSituations":"Leftover reserve cgroup with running task processes from a prior Nomad run; agent without cgroup write access; kernel lacking a controller Nomad activates (e.g. cpu controller unavailable due to cgroup_no_v1 settings or kernel version).","solutions":["Ensure nomad.slice/reserve/cgroup.procs is empty before starting Nomad (stop tasks/agent first).","Run as root or with systemd cgroup delegation so subtree_control is writable.","Verify required controllers appear in /sys/fs/cgroup/cgroup.controllers; upgrade kernel or boot config if absent.","Check the cgroup mount is not read-only."],"exampleFix":"# before: EBUSY due to live processes\n$ cat /sys/fs/cgroup/nomad.slice/reserve/cgroup.procs\n4321\n// after: stop nomad, drain tasks, then restart\n$ sudo systemctl stop nomad; kill 4321; sudo systemctl start nomad","handlingStrategy":"retry","validationCode":"func reserveIsEmpty(reserveDir string) (bool, error) {\n  b, err := os.ReadFile(filepath.Join(reserveDir, \"cgroup.procs\"))\n  if err != nil { return false, err }\n  return len(strings.TrimSpace(string(b))) == 0, nil\n}","typeGuard":"func controllersAvailable(needed []string) bool {\n  b, _ := os.ReadFile(\"/sys/fs/cgroup/cgroup.controllers\")\n  have := strings.Fields(string(b))\n  for _, n := range needed {\n    if !slices.Contains(have, n) { return false }\n  }\n  return true\n}","tryCatchPattern":"if err := cgroupslib.Init(logger, cores); err != nil {\n  if strings.Contains(err.Error(), \"subtree control on cpuset reserve\") {\n    // likely EBUSY from live task processes; drain then retry\n    return retryAfterDrain(err)\n  }\n  return err\n}","preventionTips":["Stop tasks and the agent before re-running Init so reserve cgroup.procs is empty.","Confirm all activated controllers are present at the v2 root (cgroup.controllers).","Use systemd unified hierarchy (cgroup v2) with required controllers enabled at boot.","Grant write access via root or Delegate=yes."],"tags":["linux","cgroups","cgroupv2","subtree-control"],"backgroundTag":"cgroup-write-permission-denied","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}