apache/hadoop · error · IllegalStateException

{} free ports could not be acquired.

Error message

{} free ports could not be acquired.

What it means

IllegalStateException from NetUtils.getFreeSocketPorts(numOfPorts) when, after up to numOfPorts*5 attempts, fewer than numOfPorts distinct non-zero free ports were collected. Each attempt opens a ServerSocket(0), records the port, and closes it — ports can repeat (deduped by the Set) or come back 0 on failure, so a heavily loaded host or a nearly exhausted ephemeral range starves the collection. numOfPorts must also satisfy 0 < numOfPorts <= 25 or Preconditions fails first with a different message.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java:1173

   *
   * @param numOfPorts Number of free ports to acquire.
   * @return Free ports for binding a local socket.
   */
  public static Set<Integer> getFreeSocketPorts(int numOfPorts) {
    Preconditions.checkArgument(numOfPorts > 0 && numOfPorts <= 25,
        "Valid range for num of ports is between 0 and 26");
    final Set<Integer> freePorts = new HashSet<>(numOfPorts);
    for (int i = 0; i < numOfPorts * 5; i++) {
      int port = getFreeSocketPort();
      if (port == 0) {
        continue;
      }
      freePorts.add(port);
      if (freePorts.size() == numOfPorts) {
        return freePorts;
      }
    }
    throw new IllegalStateException(numOfPorts + " free ports could not be acquired.");
  }

  /**
   * Return an @{@link InetAddress} to bind to. If bindWildCardAddress is true
   * then returns null.
   *
   * @param localAddr local addr.
   * @param bindWildCardAddress bind wildcard address.
   * @return InetAddress
   */
  public static InetAddress bindToLocalAddress(InetAddress localAddr, boolean
      bindWildCardAddress) {
    if (!bindWildCardAddress) {
      return localAddr;
    }
    return null;
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Reduce numOfPorts or run fewer concurrent port-grabbing processes
2. Widen the ephemeral range / raise fd limits on the host: sysctl net.ipv4.ip_local_port_range, ulimit -n
3. Retry with backoff — port availability is transient; or fall back to acquiring ports one at a time as needed instead of in bulk
4. Check for fd leaks (lsof | wc -l) if ServerSocket(0) keeps failing

Example fix

// before
Set<Integer> ports = NetUtils.getFreeSocketPorts(25); // loaded CI box -> IllegalStateException: 25 free ports could not be acquired.

// after
Set<Integer> ports = NetUtils.getFreeSocketPorts(5); // fewer ports per run,
// plus raise limits: sysctl -w net.ipv4.ip_local_port_range="1024 65535"; ulimit -n 65536

if (numOfPorts <= 0 || numOfPorts > 25) throw new IllegalArgumentException("numOfPorts must be in (0,25]");
long fdUse = ...; // optional: check ulimit / ephemeral range before bulk requests

catch (IllegalStateException e) { /* 'N free ports could not be acquired.' */ back off and retry with the same or smaller count; if persistent, widen ip_local_port_range / raise fd limit; }

When it happens

Trigger: Requesting many ports (e.g., 25) on a machine whose ephemeral range is mostly occupied; concurrent callers racing for the same few free ports so duplicates dominate the 5x attempts; ulimit/file-descriptor pressure making ServerSocket(0) fail repeatedly (port returns 0).

Common situations: Integration tests spawning many services in parallel on CI boxes; containers with tiny ip_local_port_range; fd exhaustion (ulimit -n) causing repeated bind failures; requesting ports while a stress test holds thousands of connections.

Related errors

• Bad format: {}
• Wrong FS: {path} and port={thatPort}, expected: {this.getUri
• src.toString() + ": No such file or directory"
• Range starts beyond the file length ({l}): {last}
• getXAttrs on path `{}' is not within a mount point

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a4db15ef8474ecb8. Report an issue: GitHub.