gRPC port range is exhausted; reusing a port that belonged t

Error message

gRPC port range is exhausted; reusing a port that belonged to another backend. A stale controller row for the previous owner could briefly misroute to this backend — raise LOCALAI_GRPC_MAX_PORT to restore headroom

What it means

Warning emitted by the backend supervisor's port allocator when the LOCALAI_GRPC min-max gRPC port range has no never-handed-out ports left, so it pops a port from the free pool that previously belonged to a different backend and reassigns it. The danger called out in the message: if the controller still has a stale row pointing the old owner's address at this port, requests can briefly route to the wrong backend. Allocation still succeeds; this is a degraded-mode heads-up, not a failure.

Source

Thrown at core/services/worker/supervisor.go:300

		if _, owned := owners[port]; owned {
			continue
		}
		s.freePorts = slices.Delete(s.freePorts, i, i+1)
		return s.claimPort(key, port), nil
	}

	// 3. Grow into ports never handed out, staying inside the range.
	if s.nextPort >= minPort && s.nextPort <= maxPort {
		port := s.nextPort
		s.nextPort++
		return s.claimPort(key, port), nil
	}

	// 4. Steal another key's port rather than refuse to start a backend.
	if len(s.freePorts) > 0 {
		port := s.freePorts[len(s.freePorts)-1]
		s.freePorts = s.freePorts[:len(s.freePorts)-1]
		xlog.Warn("gRPC port range is exhausted; reusing a port that belonged to another backend. A stale controller row for the previous owner could briefly misroute to this backend — raise LOCALAI_GRPC_MAX_PORT to restore headroom",
			"backend", key, "port", port, "previousOwner", owners[port], "min", minPort, "max", maxPort)
		return s.claimPort(key, port), nil
	}

	return 0, fmt.Errorf("%w: %d-%d is fully consumed by %d running backend(s) and %d port(s) still in quarantine; raise LOCALAI_GRPC_MAX_PORT to widen the range",
		ErrNoFreePort, minPort, maxPort, len(s.processes), len(s.quarantinedPorts))
}

// sweepAffinity drops claims whose window has lapsed, so their ports become
// ordinary free ports again. Swept lazily on allocation for the same reason as
// sweepQuarantine: the only observer is allocation itself, so a timer goroutine
// per released port would buy nothing. Callers must hold s.mu.
func (s *backendSupervisor) sweepAffinity() {
	if len(s.portAffinity) == 0 {
		return
	}
	now := time.Now()
	for key, own := range s.portAffinity {

View on GitHub (pinned to 44413a9d06)

Solutions

Raise LOCALAI_GRPC_MAX_PORT (and keep LOCALAI_GRPC_MIN_PORT sane) so nextPort growth plus quarantine headroom covers your peak concurrent backend count.
Reduce simultaneous backend count: unload models you are not using or lower concurrency limits so fewer gRPC processes live at once.
If misrouting is suspected, restart the LocalAI process (or the affected backends) so controller rows and port ownership resynchronize, then re-issue requests.
Monitor this warning in logs and size the range at (max concurrent backends) + margin for quarantined ports.

Example fix

# before
export LOCALAI_GRPC_MIN_PORT=50000
export LOCALAI_GRPC_MAX_PORT=50010   # 10 ports, exhausted with >10 backend churn

# after
export LOCALAI_GRPC_MIN_PORT=50000
export LOCALAI_GRPC_MAX_PORT=50100   # headroom for concurrent backends + quarantine

Defensive patterns

Strategy: validation

Validate before calling

# Before starting more backends, compare configured range size against
# running backends + quarantine headroom (bash + localai admin API)
MIN=${LOCALAI_GRPC_MIN_PORT:-50000}
MAX=${LOCALAI_GRPC_MAX_PORT:-50100}
RUNNING=$(curl -s localhost:8080/backends | jq 'length')
if [ $((MAX - MIN + 1)) -lt $((RUNNING + 10)) ]; then
  echo "gRPC port headroom low; raise LOCALAI_GRPC_MAX_PORT" >&2
fi

Prevention

Size LOCALAI_GRPC range to peak concurrent backends plus a quarantine margin (e.g. 2x backends + 20).
Alert on the 'gRPC port range is exhausted' warning log so you widen the range before ErrNoFreePort failures appear.
Unload idle models to keep live backend count well under range capacity.

When it happens

Trigger: Running enough concurrent backends that s.nextPort walks past maxPort (step 3 fails) while freePorts is non-empty — e.g. several models loaded and one restarting — so the allocator takes step 4, steals freePorts[len-1], logs the warning with the previous owner, and claimPort()s it for the new key.

Common situations: Default LOCALAI_GRPC port range too small for the number of model shards/replicas; frequent backend restarts churning ports; single-box dev setups with many small models; misconfigured min/max env vars leaving only a handful of ports.

Related errors

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/1ce61756f8a9beb5. Report an issue: GitHub.