hashicorp/nomad · error

failed to set subtree control on cpuset share partition: %w

Error message

failed to set subtree control on cpuset share partition: %w

What it means

After creating nomad.slice/share, Init() writes the controller activation list to its cgroup.subtree_control so task-level child cgroups under share can use the controllers. A write failure here leaves the share partition non-functional for spawning task cgroups.

Source

Thrown at client/lib/cgroupslib/init.go:172

			return fmt.Errorf("failed to set subtree control on nomad cgroup: %w", err)
		}

		if err := writeCG(cores, NomadCgroupParent, cpusetFile); err != nil {
			return fmt.Errorf("failed to write root partition cpuset: %w", err)
		}

		log.Debug("top level partition root nomad.slice cgroup initialized")

		//
		// configuring nomad.slice/share (member)
		//

		if err := mkCG(NomadCgroupParent, SharePartition()); err != nil {
			return fmt.Errorf("failed to create share cgroup: %w", err)
		}

		if err := writeCG(activation, NomadCgroupParent, SharePartition(), subtreeFile); err != nil {
			return fmt.Errorf("failed to set subtree control on cpuset share partition: %w", err)
		}

		log.Debug("partition member nomad.slice/share cgroup initialized")

		//
		// configuring nomad.slice/reserve (member)
		//

		if err := mkCG(NomadCgroupParent, ReservePartition()); err != nil {
			return fmt.Errorf("failed to create share cgroup: %w", err)
		}

		if err := writeCG(activation, NomadCgroupParent, ReservePartition(), subtreeFile); err != nil {
			return fmt.Errorf("failed to set subtree control on cpuset reserve partition: %w", err)
		}

		log.Debug("partition member nomad.slice/reserve cgroup initialized")
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Stop tasks/Nomad, verify nomad.slice/share/cgroup.procs is empty, then retry Init (EBUSY only occurs with active processes).
2. Run with sufficient privileges or systemd Delegate=yes so subtree_control is writable.
3. Check /sys/fs/cgroup/cgroup.controllers contains the controllers Nomad needs; upgrade kernel or adjust config if missing.
4. Boot with required cgroup controllers enabled (cgroup_no_v1 / systemd unified hierarchy for v2).

Example fix

# before: stale processes block subtree_control
$ cat /sys/fs/cgroup/nomad.slice/share/cgroup.procs
1234
// after: drain and clear the cgroup first
$ sudo systemctl stop nomad && kill 1234; sudo systemctl start nomad

func shareIsEmpty(shareDir string) (bool, error) {
  b, err := os.ReadFile(filepath.Join(shareDir, "cgroup.procs"))
  if err != nil { return false, err }
  return len(strings.TrimSpace(string(b))) == 0, nil
}

func subtreeWritable(dir string) bool {
  return syscall.Access(filepath.Join(dir, "cgroup.subtree_control"), syscall.W_OK) == nil
}

if err := cgroupslib.Init(logger, cores); err != nil {
  if strings.Contains(err.Error(), "subtree control on cpuset share") {
    // EBUSY: drain tasks under nomad.slice/share, then retry once
    return retryAfterDrain(err)
  }
  return err
}

When it happens

Trigger: writeCG(activation, NomadCgroupParent, SharePartition(), subtreeFile) fails — EACCES on the file, EBUSY because processes already exist inside nomad.slice/share making controller re-enable fail, or a requested controller is not available.

Common situations: Stale nomad.slice/share from a previous Nomad run still containing running processes; agent without cgroup write privileges; host kernel missing a controller Nomad tries to activate (e.g. no memory controller in cgroup.controllers).

Related errors

• failed to set subtree control on cpuset reserve partition: %
• failed to create nomad cgroup: %w
• failed to set subtree control on nomad cgroup: %w
• failed to write root partition cpuset: %w
• failed to create share cgroup: %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/6fb15bea50abb5ac. Report an issue: GitHub.