abiosoft/colima · error

could not find GGUF file for model %q: %w

Error message

could not find GGUF file for model %q: %w

What it means

ServeDockerModel wraps findGGUFPath's error with this message; findGGUFPath is the function that locates or materializes /models/bundles/sha256/<hash>/model/model.gguf, so this aggregates errors 221-225 (manifest read/parse, no GGUF layer, mkdir, ln) plus a not-running runner container.

Source

Thrown at model/docker.go:221

		return fmt.Errorf("model %q has format %q, only GGUF models are supported\n"+
			"Try a GGUF version of this model (e.g., from TheBloke on HuggingFace)",
			cfg.ModelName, modelInfo.Config.Format)
	}

	modelHash := modelInfo.Hash()
	if modelHash == "" {
		return fmt.Errorf("could not determine hash for model %q", cfg.ModelName)
	}

	// Ensure docker-model-runner container is running (needed to find GGUF path)
	if err := ensureDockerModelRunner(guest); err != nil {
		return err
	}

	// Find the GGUF file path (handles both Docker registry and HuggingFace models)
	ggufPath, err := findGGUFPath(guest, modelHash)
	if err != nil {
		return fmt.Errorf("could not find GGUF file for model %q: %w", cfg.ModelName, err)
	}

	// Get container IP
	containerIP, err := getDockerModelRunnerIP(guest)
	if err != nil {
		return err
	}

	// Kill any existing socat on this port
	stopSocat(guest, cfg.Port)

	// Start socat in background to forward localhost:port → container_ip:port
	if err := startSocat(guest, cfg.Port, containerIP); err != nil {
		return fmt.Errorf("failed to start port forwarder: %w", err)
	}

	// Run llama-server interactively (blocking, with visible output)
	// Ctrl-C will be received by the interactive process directly

View on GitHub (pinned to c3a5f9184d)

Solutions

1. Read the wrapped error: it names the exact failing step (read manifest / parse / no GGUF layer / mkdir / link) — follow the matching fix
2. Ensure the runner is up: colima ssh -- docker ps | grep docker-model-runner (ensureDockerModelRunner runs before, but it can still die)
3. Re-pull the model to restore blob+manifest, then retry serve
4. Free disk space and retry if mkdir/link failed

// Pre-flight for the whole bundle path: runner running + blob present.
func bundlePrecheck(guest environment.VM, modelHash string) error {
    if err := ensureDockerModelRunner(guest); err != nil {
        return err
    }
    if guest.RunQuiet("docker", "exec", "docker-model-runner",
        "test", "-d", "/models/manifests/sha256/"+modelHash) != nil {
        return fmt.Errorf("manifest for %s missing — re-pull the model", modelHash)
    }
    return nil
}

ggufPath, err := findGGUFPath(guest, modelHash)
if err != nil {
    return fmt.Errorf("could not find GGUF file for model %q: %w — ensure runner is running and re-pull the model", cfg.ModelName, err)
}

When it happens

Trigger: Serving a model whose bundle is absent and cannot be rebuilt (missing manifest/blob); docker-model-runner container not running when findGGUFPath execs into it; non-GGUF layer set; disk full preventing mkdir/link.

Common situations: First serve of a freshly pulled model when the runner hasn't created its bundle; after disk cleanup pruned blobs; after runner reinstallation with an empty volume.

Related errors

• no models available Pull a model first: colima model pull ai
• failed to read model manifest: %w
• failed to parse model manifest: %w
• failed to create bundle directory: %w
• failed to link model file: %w

AI-assisted analysis of abiosoft/colima@c3a5f9184d (2026-08-15). Data as JSON: /api/errors/0baa92bb4b8ba83c. Report an issue: GitHub.