abiosoft/colima · error

no GGUF layer found in model manifest

Error message

no GGUF layer found in model manifest

What it means

After parsing the manifest, findGGUFPath scans manifest.Layers for one whose MediaType contains "gguf" and whose Digest carries the "sha256:" prefix. If no layer matched, blobPath stays empty and this error is returned: the model image contains no GGUF layer (or uses digests this code cannot map to /models/blobs/sha256/<hash>).

Source

Thrown at model/docker.go:81

	var manifest ociManifest
	if err := json.Unmarshal([]byte(output), &manifest); err != nil {
		return "", fmt.Errorf("failed to parse model manifest: %w", err)
	}

	// Find the GGUF layer (mediaType contains "gguf")
	var blobPath string
	for _, layer := range manifest.Layers {
		if strings.Contains(layer.MediaType, "gguf") {
			if blobHash, ok := strings.CutPrefix(layer.Digest, "sha256:"); ok {
				blobPath = fmt.Sprintf("/models/blobs/sha256/%s", blobHash)
				break
			}
		}
	}

	if blobPath == "" {
		return "", fmt.Errorf("no GGUF layer found in model manifest")
	}

	// Create bundle directory and hard-link the blob (same approach as Docker Model Runner)
	bundleDir := fmt.Sprintf("/models/bundles/sha256/%s/model", modelHash)
	if err := guest.RunQuiet("docker", "exec", "docker-model-runner", "mkdir", "-p", bundleDir); err != nil {
		return "", fmt.Errorf("failed to create bundle directory: %w", err)
	}

	if err := guest.RunQuiet("docker", "exec", "docker-model-runner", "ln", blobPath, bundlePath); err != nil {
		return "", fmt.Errorf("failed to link model file: %w", err)
	}

	return bundlePath, nil
}

// InspectDockerModel returns information about a Docker model.
func InspectDockerModel(modelName string) (*DockerModelInfo, error) {
	guest := lima.New(host.New())

View on GitHub (pinned to c3a5f9184d)

Solutions

1. Pick an explicit GGUF variant of the model (e.g. TheBloke GGUF repos on HuggingFace, or *-gguf tags)
2. Verify what the manifest actually contains: docker exec docker-model-runner cat /models/manifests/sha256/<hash> | jq '.layers[].mediaType'
3. docker model rm the offending model, pull the GGUF variant, retry serve
4. Update Docker Model Runner / colima in case the layer mediaType scheme changed

// Before serving, verify the pulled model carries a GGUF layer.
func manifestHasGGUFLayer(manifestJSON []byte) bool {
    var m struct{ Layers []struct{ MediaType, Digest string } }
    _ = json.Unmarshal(manifestJSON, &m)
    for _, l := range m.Layers {
        if strings.Contains(l.MediaType, "gguf") && strings.HasPrefix(l.Digest, "sha256:") {
            return true
        }
    }
    return false
}

if blobPath == "" {
    return "", fmt.Errorf("no GGUF layer found in model manifest — pull a GGUF variant of this model")
}

When it happens

Trigger: Pulling a model shipped as safetensors or another non-GGUF format; a manifest whose layer digests use a different algorithm (no sha256: prefix); a manifest-list/index with no direct GGUF layer entries.

Common situations: HuggingFace models that default to safetensors; model variants named identically to GGUF ones but with different artifacts; newer runner manifest layouts.

Related errors

• model %q has format %q, only GGUF models are supported Try a
• failed to read model manifest: %w
• failed to parse model manifest: %w
• could not find GGUF file for model %q: %w

AI-assisted analysis of abiosoft/colima@c3a5f9184d (2026-08-15). Data as JSON: /api/errors/c875a6fe7f5d845a. Report an issue: GitHub.