{"record":{"id":"9bbc73389c7d7c3f","repo":"abiosoft/colima","slug":"model-q-has-format-q-only-gguf-models-are-suppo","errorCode":null,"errorMessage":"model %q has format %q, only GGUF models are supported\nTry a GGUF version of this model (e.g., from TheBloke on HuggingFace)","messagePattern":"model %q has format %q, only GGUF models are supported\nTry a GGUF version of this model \\(e\\.g\\., from TheBloke on HuggingFace\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"model/docker.go","lineNumber":203,"sourceCode":"\tguest := lima.New(host.New())\n\n\t// Set defaults\n\tif cfg.Threads <= 0 {\n\t\tcfg.Threads = 2\n\t}\n\tif cfg.GPULayers <= 0 {\n\t\tcfg.GPULayers = 999\n\t}\n\n\t// Get the model info (model should already be available via EnsureDockerModel)\n\tmodelInfo, err := InspectDockerModel(cfg.ModelName)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to inspect model %q: %w\", cfg.ModelName, err)\n\t}\n\n\t// Check model format - only GGUF models are supported\n\tif modelInfo.Config.Format != \"gguf\" {\n\t\treturn fmt.Errorf(\"model %q has format %q, only GGUF models are supported\\n\"+\n\t\t\t\"Try a GGUF version of this model (e.g., from TheBloke on HuggingFace)\",\n\t\t\tcfg.ModelName, modelInfo.Config.Format)\n\t}\n\n\tmodelHash := modelInfo.Hash()\n\tif modelHash == \"\" {\n\t\treturn fmt.Errorf(\"could not determine hash for model %q\", cfg.ModelName)\n\t}\n\n\t// Ensure docker-model-runner container is running (needed to find GGUF path)\n\tif err := ensureDockerModelRunner(guest); err != nil {\n\t\treturn err\n\t}\n\n\t// Find the GGUF file path (handles both Docker registry and HuggingFace models)\n\tggufPath, err := findGGUFPath(guest, modelHash)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"could not find GGUF file for model %q: %w\", cfg.ModelName, err)","sourceCodeStart":185,"sourceCodeEnd":221,"githubUrl":"https://github.com/abiosoft/colima/blob/c3a5f9184d83a197184f897a9f07eb3c01b3bc88/model/docker.go#L185-L221","documentation":"ServeDockerModel only serves GGUF models: llama-server consumes a .gguf file, so modelInfo.Config.Format != \"gguf\" is rejected with guidance to pick a GGUF variant. The model resolved by inspect is real and local, but its artifact format (safetensors, ggml, onnx, etc.) cannot be served.","triggerScenarios":"Serving an HF model that defaults to safetensors; a ggml-era model pulled by its plain name; registry tag that points at a non-GGUF build; multi-artifact repos where inspect reports the primary (non-GGUF) format.","commonSituations":"Users typing a popular model name without choosing the GGUF build; models pulled for other runtimes (ollama-era ggml, transformers safetensors) reused here.","solutions":["Pull and serve an explicit GGUF variant (e.g. TheBloke <model>-GGUF repos on HuggingFace)","docker model rm the non-GGUF copy, ensure the GGUF one, then serve","Confirm the format before serving: colima ssh -- docker model inspect <name> and check the format field","If you believe the model IS gguf, update Docker Model Runner/colima in case the reported format string changed"],"exampleFix":"# before\ncolima model serve <model>        # safetensors build -> error\n\n# after\ncolima model pull huggingface.co/TheBloke/<model>-GGUF:latest\ncolima model serve huggingface.co/TheBloke/<model>-GGUF:latest","handlingStrategy":"type-guard","validationCode":"info, err := InspectDockerModel(name)\nif err == nil && info.Config.Format != \"gguf\" {\n    // pick the GGUF variant before attempting serve\n}","typeGuard":"func isServableGGUF(info *DockerModelInfo) bool {\n    return info != nil && strings.EqualFold(info.Config.Format, \"gguf\")\n}","tryCatchPattern":"if !isServableGGUF(modelInfo) {\n    return fmt.Errorf(\"model %q has format %q, only GGUF models are supported — pull a GGUF variant (e.g. TheBloke on HuggingFace)\", cfg.ModelName, modelInfo.Config.Format)\n}","preventionTips":["Check inspect's format field immediately after pulling a new model","Standardize on explicit GGUF repos/tags in team docs so users don't grab safetensors builds","Fail fast on format before any bundle/port setup runs"],"tags":["gguf","model-format","docker","validation"],"backgroundTag":null,"analyzedSha":"c3a5f9184d83a197184f897a9f07eb3c01b3bc88","analyzedAt":"2026-08-15T18:58:08.334Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}