hashicorp/nomad · error

bandwidth exceeded

Error message

bandwidth exceeded

What it means

Inside AssignTaskNetwork's yieldIP callback, the ask's MBits plus the device's already-used bandwidth exceeds idx.AvailBandwidth for that device, so the candidate network is rejected with "bandwidth exceeded". This is the deprecated task-network path's bandwidth reservation check.

Source

Thrown at nomad/structs/network.go:586

// AssignTaskNetwork is used to offer network resources given a
// task.resources.network ask.  If the ask cannot be satisfied, returns nil
//
// AssignTaskNetwork and task.resources.network are deprecated in favor of
// AssignPorts and group.network. AssignTaskNetwork does not support multiple
// interfaces and only uses the node's default interface. AssignPorts is the
// method that is used for group.network asks.
func (idx *NetworkIndex) AssignTaskNetwork(ask *NetworkResource) (out *NetworkResource, err error) {
	err = fmt.Errorf("no networks available")
	idx.yieldIP(func(n *NetworkResource, offerIP net.IP) (stop bool) {
		// Convert the IP to a string
		offerIPStr := offerIP.String()

		// Check if we would exceed the bandwidth cap
		availBandwidth := idx.AvailBandwidth[n.Device]
		usedBandwidth := idx.UsedBandwidth[n.Device]
		if usedBandwidth+ask.MBits > availBandwidth {
			err = fmt.Errorf("bandwidth exceeded")
			return
		}

		used := idx.UsedPorts[offerIPStr]

		// Check if any of the reserved ports are in use
		for _, port := range ask.ReservedPorts {
			// Guard against invalid port
			if port.Value < 0 || port.Value >= MaxValidPort {
				err = fmt.Errorf("invalid port %d (out of range)", port.Value)
				return
			}

			// Check if in use
			if used != nil && used.Check(uint(port.Value)) {
				err = fmt.Errorf("reserved port collision %s=%d", port.Label, port.Value)
				return
			}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Lower the task's resources.network.mbits request to fit available device bandwidth.
2. Fix the client's link speed detection / set interface speed so AvailBandwidth is correct.
3. Schedule on a different node or device with free bandwidth.
4. Migrate to group network mode = "bridge", which does not consume device MBits reservations this way.

Example fix

// before (job HCL)
resources { network { mbits = 500 } }
// after
resources { network { mbits = 50 } } // within device AvailBandwidth

// keep ask.MBits within the device's advertised capacity
if ask.MBits > nodeAvailMBits(device) {
  return fmt.Errorf("ask %d MBits exceeds %d available on %s", ask.MBits, nodeAvailMBits(device), device)
}

func fitsBandwidth(ask, used, avail int) bool { return used+ask <= avail }

null

When it happens

Trigger: AssignTaskNetwork called with ask.MBits such that usedBandwidth[device] + ask.MBits > availBandwidth[device] for the yielded network device.

Common situations: Node link speed misdetected (e.g., 0 or 100 Mbps link), many tasks reserving MBits on one device, job requesting more MBits than the interface's total capacity.

Related errors

• no networks available
• network namespace already exists but was misconfigured
• network already configured but not found in state
• no CNI network config found
• ErrConnectRequireOneNetwork

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c4488b5d8fc3ef00. Report an issue: GitHub.