hashicorp/nomad · error

failed to set subtree control on nomad cgroup: %w

Error message

failed to set subtree control on nomad cgroup: %w

What it means

During cgroups v2 client initialization, Nomad creates its top-level nomad.slice cgroup and writes the controller activation list (+cpuset +memory, etc.) to its cgroup.subtree_control file. This error wraps any failure writing that file, meaning Nomad cannot enable controllers for child cgroups and task isolation will not work.

Source

Thrown at client/lib/cgroupslib/init.go:154

		// clients with delegated cgroups typically won't be able to write to
		// the subtree file, but that's ok so long as the required controllers
		// are activated
		if !functionalCgroups2(subtreeFile) {
			if err := writeCG(activation, subtreeFile); err != nil {
				return fmt.Errorf("failed to create nomad cgroup: %w", err)
			}
		}

		//
		// configuring nomad.slice
		//

		if err := mkCG(NomadCgroupParent); err != nil {
			return fmt.Errorf("failed to create nomad cgroup: %w", err)
		}

		if err := writeCG(activation, NomadCgroupParent, subtreeFile); err != nil {
			return fmt.Errorf("failed to set subtree control on nomad cgroup: %w", err)
		}

		if err := writeCG(cores, NomadCgroupParent, cpusetFile); err != nil {
			return fmt.Errorf("failed to write root partition cpuset: %w", err)
		}

		log.Debug("top level partition root nomad.slice cgroup initialized")

		//
		// configuring nomad.slice/share (member)
		//

		if err := mkCG(NomadCgroupParent, SharePartition()); err != nil {
			return fmt.Errorf("failed to create share cgroup: %w", err)
		}

		if err := writeCG(activation, NomadCgroupParent, SharePartition(), subtreeFile); err != nil {
			return fmt.Errorf("failed to set subtree control on cpuset share partition: %w", err)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Run the Nomad agent as root (or with CAP_SYS_ADMIN / appropriate cgroup delegation) so it can write to /sys/fs/cgroup/nomad.slice/cgroup.subtree_control.
2. Check that the needed controllers are listed in /sys/fs/cgroup/cgroup.controllers and not already enabled in child cgroups (EBUSY); disable conflicting children or reboot.
3. Verify /sys/fs/cgroup is mounted rw (mount | grep cgroup); remount without 'ro' if needed.
4. If using systemd cgroup delegation, delegate the subtree to the Nomad service (Delegate=yes) and configure Nomad to use the delegated path.

Example fix

// before
sudo nomad agent -config /etc/nomad.d
// after (systemd unit with cgroup delegation)
[Service]
ExecStart=/usr/bin/nomad agent -config /etc/nomad.d
Delegate=yes
CapabilityBoundingSet=CAP_SYS_ADMIN

func canWriteSubtreeControl() error {
  b, err := os.ReadFile("/sys/fs/cgroup/cgroup.controllers")
  if err != nil { return err }
  if len(b) == 0 { return errors.New("no controllers available at root cgroup") }
  return syscall.Access("/sys/fs/cgroup/nomad.slice", syscall.W_OK)
}

func cgroupWritable(path string) bool {
  return syscall.Access(filepath.Join(path, "cgroup.subtree_control"), syscall.W_OK) == nil
}

if err := cgroupslib.Init(logger, cores); err != nil {
  var perr *fs.PathError
  if errors.As(err, &perr) && errors.Is(perr.Err, syscall.EACCES) {
    logger.Error("run nomad as root or delegate cgroups (systemd Delegate=yes)", "path", perr.Path)
  }
  return err
}

When it happens

Trigger: Init() is called on a cgroups v2 host and os.WriteFile on /sys/fs/cgroup/nomad.slice/cgroup.subtree_control fails — typically EACCES (process lacks write access to the root cgroup), EBUSY (a controller already enabled in a child blocks activation), or the nomad.slice directory was created but is read-only.

Common situations: Nomad agent run as non-root or in a container without cgroup write privileges; systemd owns the root cgroup and delegates only a subtree to Nomad (delegated cgroups); cgroup filesystem mounted read-only; controllers not available in /sys/fs/cgroup/cgroup.controllers so activation fails.

Related errors

• failed to write root partition cpuset: %w
• failed to create share cgroup: %w
• failed to set subtree control on cpuset share partition: %w
• failed to set subtree control on cpuset reserve partition: %
• cgroupslib: unable to update share cpuset with %q: %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/ebc14352a0dd8359. Report an issue: GitHub.