mudler/LocalAI · error · ErrNoFreePort

%w: %d-%d is fully consumed by %d running backend(s) and %d

Error message

%w: %d-%d is fully consumed by %d running backend(s) and %d port(s) still in quarantine; raise LOCALAI_GRPC_MAX_PORT to widen the range

What it means

Hard allocation failure from the same supervisor: the gRPC port range is fully consumed — every port is either held by one of len(s.processes) running backends or sitting in len(s.quarantinedPorts) quarantine (ports held back after a backend died, pending a sweep). The returned error wraps ErrNoFreePort with the exact counts, so Go callers can match it with errors.Is. Backend startup is refused.

Source

Thrown at core/services/worker/supervisor.go:305

	}

	// 3. Grow into ports never handed out, staying inside the range.
	if s.nextPort >= minPort && s.nextPort <= maxPort {
		port := s.nextPort
		s.nextPort++
		return s.claimPort(key, port), nil
	}

	// 4. Steal another key's port rather than refuse to start a backend.
	if len(s.freePorts) > 0 {
		port := s.freePorts[len(s.freePorts)-1]
		s.freePorts = s.freePorts[:len(s.freePorts)-1]
		xlog.Warn("gRPC port range is exhausted; reusing a port that belonged to another backend. A stale controller row for the previous owner could briefly misroute to this backend — raise LOCALAI_GRPC_MAX_PORT to restore headroom",
			"backend", key, "port", port, "previousOwner", owners[port], "min", minPort, "max", maxPort)
		return s.claimPort(key, port), nil
	}

	return 0, fmt.Errorf("%w: %d-%d is fully consumed by %d running backend(s) and %d port(s) still in quarantine; raise LOCALAI_GRPC_MAX_PORT to widen the range",
		ErrNoFreePort, minPort, maxPort, len(s.processes), len(s.quarantinedPorts))
}

// sweepAffinity drops claims whose window has lapsed, so their ports become
// ordinary free ports again. Swept lazily on allocation for the same reason as
// sweepQuarantine: the only observer is allocation itself, so a timer goroutine
// per released port would buy nothing. Callers must hold s.mu.
func (s *backendSupervisor) sweepAffinity() {
	if len(s.portAffinity) == 0 {
		return
	}
	now := time.Now()
	for key, own := range s.portAffinity {
		if own.expired(now) {
			delete(s.portAffinity, key)
		}
	}
}

View on GitHub (pinned to 44413a9d06)

Solutions

Raise LOCALAI_GRPC_MAX_PORT (the message's own remedy) to widen the range, then retry loading the model.
Free ports by unloading/stopping backends you do not need right now.
If quarantine is the culprit (the error prints the count), wait for the quarantine window to lapse and be swept on the next allocation, or restart LocalAI to reset allocator state.
Set the range to at least (max concurrent backends + expected quarantine burst) in your service unit/env before starting LocalAI.

Example fix

# before
LOCALAI_GRPC_MAX_PORT=50016  # fully consumed by 14 running backend(s) and 2 port(s) still in quarantine

# after
LOCALAI_GRPC_MAX_PORT=50100  # wide enough for peak backends plus quarantine headroom

Defensive patterns

Strategy: retry

Try / catch

port, err := supervisor.StartBackend(key)
if err != nil {
    if errors.Is(err, worker.ErrNoFreePort) {
        // free backends or wait out quarantine, then retry once headroom exists
        time.Sleep(quarantineWindow)
        port, err = supervisor.StartBackend(key)
    }
    if err != nil {
        return fmt.Errorf("cannot start backend %s: %w", key, err)
    }
}

Prevention

Set LOCALAI_GRPC_MAX_PORT from your max-concurrent-backend SLO, not the default.
Parse the running/quarantined counts out of the error to decide between waiting (quarantine-heavy) and widening (capacity-heavy).
Restart LocalAI to reset quarantine state after a crash loop, then re-check the range size.

When it happens

Trigger: Requesting a new backend when s.nextPort is outside [minPort,maxPort] AND freePorts is empty — i.e. running backend count + quarantined ports covers the whole range. Typical with a tight default range, several loaded models, and recent crashes whose ports are still quarantined.

Common situations: Crash-looping backends filling quarantine; a small LOCALAI_GRPC port range on a busy box; loading a new model when the range was sized for fewer replicas; stale supervisor state after many restarts within a short window.

Related errors

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/3b865ab75fc30531. Report an issue: GitHub.