{"record":{"id":"0baa92bb4b8ba83c","repo":"abiosoft/colima","slug":"could-not-find-gguf-file-for-model-q-w","errorCode":null,"errorMessage":"could not find GGUF file for model %q: %w","messagePattern":"could not find GGUF file for model %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"model/docker.go","lineNumber":221,"sourceCode":"\t\treturn fmt.Errorf(\"model %q has format %q, only GGUF models are supported\\n\"+\n\t\t\t\"Try a GGUF version of this model (e.g., from TheBloke on HuggingFace)\",\n\t\t\tcfg.ModelName, modelInfo.Config.Format)\n\t}\n\n\tmodelHash := modelInfo.Hash()\n\tif modelHash == \"\" {\n\t\treturn fmt.Errorf(\"could not determine hash for model %q\", cfg.ModelName)\n\t}\n\n\t// Ensure docker-model-runner container is running (needed to find GGUF path)\n\tif err := ensureDockerModelRunner(guest); err != nil {\n\t\treturn err\n\t}\n\n\t// Find the GGUF file path (handles both Docker registry and HuggingFace models)\n\tggufPath, err := findGGUFPath(guest, modelHash)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"could not find GGUF file for model %q: %w\", cfg.ModelName, err)\n\t}\n\n\t// Get container IP\n\tcontainerIP, err := getDockerModelRunnerIP(guest)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Kill any existing socat on this port\n\tstopSocat(guest, cfg.Port)\n\n\t// Start socat in background to forward localhost:port → container_ip:port\n\tif err := startSocat(guest, cfg.Port, containerIP); err != nil {\n\t\treturn fmt.Errorf(\"failed to start port forwarder: %w\", err)\n\t}\n\n\t// Run llama-server interactively (blocking, with visible output)\n\t// Ctrl-C will be received by the interactive process directly","sourceCodeStart":203,"sourceCodeEnd":239,"githubUrl":"https://github.com/abiosoft/colima/blob/c3a5f9184d83a197184f897a9f07eb3c01b3bc88/model/docker.go#L203-L239","documentation":"ServeDockerModel wraps findGGUFPath's error with this message; findGGUFPath is the function that locates or materializes /models/bundles/sha256/<hash>/model/model.gguf, so this aggregates errors 221-225 (manifest read/parse, no GGUF layer, mkdir, ln) plus a not-running runner container.","triggerScenarios":"Serving a model whose bundle is absent and cannot be rebuilt (missing manifest/blob); docker-model-runner container not running when findGGUFPath execs into it; non-GGUF layer set; disk full preventing mkdir/link.","commonSituations":"First serve of a freshly pulled model when the runner hasn't created its bundle; after disk cleanup pruned blobs; after runner reinstallation with an empty volume.","solutions":["Read the wrapped error: it names the exact failing step (read manifest / parse / no GGUF layer / mkdir / link) — follow the matching fix","Ensure the runner is up: colima ssh -- docker ps | grep docker-model-runner (ensureDockerModelRunner runs before, but it can still die)","Re-pull the model to restore blob+manifest, then retry serve","Free disk space and retry if mkdir/link failed"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Pre-flight for the whole bundle path: runner running + blob present.\nfunc bundlePrecheck(guest environment.VM, modelHash string) error {\n    if err := ensureDockerModelRunner(guest); err != nil {\n        return err\n    }\n    if guest.RunQuiet(\"docker\", \"exec\", \"docker-model-runner\",\n        \"test\", \"-d\", \"/models/manifests/sha256/\"+modelHash) != nil {\n        return fmt.Errorf(\"manifest for %s missing — re-pull the model\", modelHash)\n    }\n    return nil\n}","typeGuard":null,"tryCatchPattern":"ggufPath, err := findGGUFPath(guest, modelHash)\nif err != nil {\n    return fmt.Errorf(\"could not find GGUF file for model %q: %w — ensure runner is running and re-pull the model\", cfg.ModelName, err)\n}","preventionTips":["Run the runner/blob preflight before serve instead of relying on the bundle already existing","Keep the wrapped inner error visible; it distinguishes manifest/mkdir/link causes","Re-pull after runner reinstalls or disk cleanups to keep blob+manifest+bundle consistent"],"tags":["docker","gguf","model-runner","aggregate"],"backgroundTag":null,"analyzedSha":"c3a5f9184d83a197184f897a9f07eb3c01b3bc88","analyzedAt":"2026-08-15T18:58:08.334Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}