abiosoft/colima · error

failed to inspect model %q: %w

Error message

failed to inspect model %q: %w

What it means

ServeDockerModel starts by calling InspectDockerModel(cfg.ModelName); failure wraps into this error. Same root causes as error 226 (model absent, docker model CLI problems, runner down), plus the documented contract that callers run EnsureDockerModel first so the model exists before serving.

Source

Thrown at model/docker.go:198

// ServeDockerModel serves a Docker model with llama-server.
// It runs llama-server interactively (with visible output) and uses socat to forward the port.
// The function blocks until interrupted (Ctrl-C) or llama-server exits.
// Note: Call EnsureDockerModel first to ensure the model is available.
func ServeDockerModel(cfg DockerModelServeConfig) error {
	guest := lima.New(host.New())

	// Set defaults
	if cfg.Threads <= 0 {
		cfg.Threads = 2
	}
	if cfg.GPULayers <= 0 {
		cfg.GPULayers = 999
	}

	// Get the model info (model should already be available via EnsureDockerModel)
	modelInfo, err := InspectDockerModel(cfg.ModelName)
	if err != nil {
		return fmt.Errorf("failed to inspect model %q: %w", cfg.ModelName, err)
	}

	// Check model format - only GGUF models are supported
	if modelInfo.Config.Format != "gguf" {
		return fmt.Errorf("model %q has format %q, only GGUF models are supported\n"+
			"Try a GGUF version of this model (e.g., from TheBloke on HuggingFace)",
			cfg.ModelName, modelInfo.Config.Format)
	}

	modelHash := modelInfo.Hash()
	if modelHash == "" {
		return fmt.Errorf("could not determine hash for model %q", cfg.ModelName)
	}

	// Ensure docker-model-runner container is running (needed to find GGUF path)
	if err := ensureDockerModelRunner(guest); err != nil {
		return err
	}

View on GitHub (pinned to c3a5f9184d)

Solutions

1. Call EnsureDockerModel(cfg.ModelName) before ServeDockerModel (the function comment requires it)
2. Reproduce with colima ssh -- docker model inspect <name> and fix per error 226
3. If the model was pruned, re-pull it and retry
4. Verify docker model support in the VM (docker model version) and update if missing

Example fix

// before
err := ServeDockerModel(cfg)

// after
if _, err := EnsureDockerModel(cfg.ModelName); err != nil {
    return err
}
err := ServeDockerModel(cfg)

// Enforce ServeDockerModel's documented precondition.
if _, err := EnsureDockerModel(cfg.ModelName); err != nil {
    return err
}
err := ServeDockerModel(cfg)

modelInfo, err := InspectDockerModel(cfg.ModelName)
if err != nil {
    return fmt.Errorf("failed to inspect model %q: %w — was EnsureDockerModel called first?", cfg.ModelName, err)
}

When it happens

Trigger: Calling ServeDockerModel directly without EnsureDockerModel; serving a model removed since the ensure step; runner stopped between ensure and serve; docker model subcommand unavailable.

Common situations: Scripts that skip the ensure step to save a round trip; long-lived sessions where the model was pruned; colima VM restarted between ensure and serve.

Related errors

• no models available Pull a model first: colima model pull ai
• failed to read model manifest: %w
• failed to parse model manifest: %w
• failed to create bundle directory: %w
• failed to link model file: %w

AI-assisted analysis of abiosoft/colima@c3a5f9184d (2026-08-15). Data as JSON: /api/errors/91147318ee932c55. Report an issue: GitHub.