abiosoft/colima · error

failed to start port forwarder

Error message

failed to start port forwarder: %w

What it means

ServeDockerModel forwards the host port to the runner container by running, inside the VM, `nohup socat TCP-LISTEN:<port>,fork,reuseaddr TCP:<containerIP>:<port> &` via guest.Run("sh","-c",cmd). This error means that shell command exited non-zero: socat is not installed in the VM guest, the port is already bound (a leftover socat that stopSocat failed to kill, or another service), or sh reported an error.

Solutions

1. Kill leftovers in the VM and retry: colima ssh -- 'pkill -f "socat TCP-LISTEN:<port>"'
2. Serve on a different, likely-free port (cfg.Port)
3. Check socat exists in the guest: colima ssh -- which socat; if missing, install it in the guest or use an image that includes it
4. Verify what holds the port: colima ssh -- 'ss -ltnp | grep <port>'

// Before startSocat: confirm the guest port is free and socat exists.
func portForwardPrecheck(guest environment.VM, port int) error {
    if guest.RunQuiet("sh", "-c", fmt.Sprintf("command -v socat >/dev/null")) != nil {
        return fmt.Errorf("socat not installed in VM")
    }
    if guest.RunQuiet("sh", "-c", fmt.Sprintf("! ss -ltn | grep -q ':%d '", port)) != nil {
        return fmt.Errorf("port %d already in use in VM", port)
    }
    return nil
}

if err := startSocat(guest, cfg.Port, containerIP); err != nil {
    _ = stopSocat(guest, cfg.Port) // clear any half-started listener
    return fmt.Errorf("failed to start port forwarder: %w — is socat installed and port %d free?", err, cfg.Port)
}

When it happens

Trigger: Choosing a port already LISTENing in the VM (previous serve's socat survived stopSocat, or another daemon owns it); guest image lacking /usr/bin/socat; containerIP unreachable so socat exits immediately (less common since nohup backgrounds it).

Common situations: Repeated serves on the same port after an unclean exit; ports like 8080 that other VM services use; minimal/custom colima images without socat.

Related errors

• error reinstalling Docker Model Runner
• error setting up AI model runner
• failed to pull model
• cannot copy registry certs to vm
• 'colima model' requires docker runtime, current runtime is

AI-assisted analysis of abiosoft/colima@c3a5f9184d (2026-08-15). Data as JSON: /api/errors/6ac084b3e32772a9. Report an issue: GitHub.

Appendix: source

Thrown at model/docker.go:235

	// Find the GGUF file path (handles both Docker registry and HuggingFace models)
	ggufPath, err := findGGUFPath(guest, modelHash)
	if err != nil {
		return fmt.Errorf("could not find GGUF file for model %q: %w", cfg.ModelName, err)
	}

	// Get container IP
	containerIP, err := getDockerModelRunnerIP(guest)
	if err != nil {
		return err
	}

	// Kill any existing socat on this port
	stopSocat(guest, cfg.Port)

	// Start socat in background to forward localhost:port → container_ip:port
	if err := startSocat(guest, cfg.Port, containerIP); err != nil {
		return fmt.Errorf("failed to start port forwarder: %w", err)
	}

	// Run llama-server interactively (blocking, with visible output)
	// Ctrl-C will be received by the interactive process directly
	// Use -it for TTY, -i for non-TTY (e.g., piped or CI environments)
	execFlag := "-i"
	if terminal.IsTerminal() {
		execFlag = "-it"
	}

	err = guest.RunInteractive("docker", "exec", execFlag, "docker-model-runner",
		"/app/bin/com.docker.llama-server",
		"-ngl", fmt.Sprintf("%d", cfg.GPULayers),
		"--metrics",
		"--threads", fmt.Sprintf("%d", cfg.Threads),
		"--model", ggufPath,
		"--alias", cfg.ModelName,
		"--host", "0.0.0.0",

View on GitHub (pinned to c3a5f9184d)