{"record":{"id":"0e0c00ced47360c9","repo":"hashicorp/nomad","slug":"dynamic-port-selection-failed","errorCode":null,"errorMessage":"dynamic port selection failed","messagePattern":"dynamic port selection failed","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/structs/network.go","lineNumber":687,"sourceCode":"\t} else {\n\t\tusedSet, err = NewBitmap(MaxValidPort)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tfor _, port := range reserved {\n\t\tusedSet.Set(uint(port.Value))\n\t}\n\n\t// Get the indexes of the unset ports, less those which have already been\n\t// picked as part of this offer\n\tavailablePorts := usedSet.IndexesInRangeFiltered(\n\t\tfalse, uint(minDynamicPort), uint(maxDynamicPort), portsInOffer)\n\n\t// Randomize the amount we need\n\tif len(availablePorts) < numDyn {\n\t\treturn nil, fmt.Errorf(\"dynamic port selection failed\")\n\t}\n\n\tnumAvailable := len(availablePorts)\n\tfor i := range numDyn {\n\t\tj := rand.Intn(numAvailable)\n\t\tavailablePorts[i], availablePorts[j] = availablePorts[j], availablePorts[i]\n\t}\n\n\treturn availablePorts[:numDyn], nil\n}\n\n// getDynamicPortsStochastic takes the nodes used port bitmap which may be nil\n// if no ports have been allocated yet, any ports already offered in the caller,\n// and the network ask. It returns a set of unused ports to fulfil the ask's\n// DynamicPorts or an error if it failed. An error does not mean the ask can not\n// be satisfied as the method has a fixed amount of random probes and if these\n// fail, the search is aborted.\nfunc getDynamicPortsStochastic(nodeUsed Bitmap, portsInOffer []int, minDynamicPort, maxDynamicPort int, reservedPorts []Port, count int) ([]int, error) {","sourceCodeStart":669,"sourceCodeEnd":705,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/structs/network.go#L669-L705","documentation":"getDynamicPortsPrecise enumerates all free indexes in the dynamic port range (minus ports already in the current offer) via IndexesInRangeFiltered, then randomly samples numDyn of them. If fewer free ports than requested dynamic ports exist in [minDynamicPort, maxDynamicPort], it fails with \"dynamic port selection failed\". This means the node's dynamic port space for that address is exhausted (or over-fragmented).","triggerScenarios":"AssignPorts (or its anonymous helper) requests more dynamic ports than the count of unreserved ports available in the configured dynamic range on that address, e.g. ask.DynamicPorts length > len(availablePorts).","commonSituations":"Node with many allocations exhausting the default 20000-32000 dynamic range; a job asking for dozens of dynamic ports; a narrowed custom dynamic range via client config.","solutions":["Widen the client's dynamic port range (e.g. set dynamic_port_range = \"10000-40000\").","Reduce the number of dynamic ports the job requests.","Move the allocation to a less loaded node.","Audit for leaked/stale allocations holding ports on the node."],"exampleFix":"// before (client.hcl)\n# dynamic_port_range not set (default 20000-32000)\n// after\ndynamic_port_range = \"10000-45000\"","handlingStrategy":"fallback","validationCode":"// compare requested dynamic port count with the node's free dynamic capacity\nfree := countFreePortsInRange(node, dynamicMin, dynamicMax)\nif free < len(ask.DynamicPorts) {\n  return fmt.Errorf(\"node has %d free dynamic ports, need %d\", free, len(ask.DynamicPorts))\n}","typeGuard":"func hasCapacity(free, needed int) bool { return free >= needed }","tryCatchPattern":"null","preventionTips":["Widen dynamic_port_range on busy clients","Monitor per-node dynamic port utilization and alert before exhaustion","Avoid jobs that request very large numbers of dynamic ports"],"tags":["nomad","ports","exhaustion","scheduling"],"backgroundTag":"port-range-exhausted","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}