{"record":{"id":"9455e56b779965f3","repo":"zed-industries/zed","slug":"llama-cpp-does-not-support-custom-tools","errorCode":null,"errorMessage":"llama.cpp does not support custom tools","messagePattern":"llama\\.cpp does not support custom tools","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/language_models/src/provider/llama_cpp.rs","lineNumber":700,"sourceCode":"                request,\n                &extra_headers,\n            )\n            .await?;\n            Ok(stream)\n        });\n\n        async move { Ok(future.await?.boxed()) }.boxed()\n    }\n}\n\nfn build_llama_cpp_request(\n    model_name: &str,\n    supports_images: bool,\n    capabilities: LiveCapabilities,\n    request: LanguageModelRequest,\n) -> Result<llama_cpp::ChatCompletionRequest> {\n    if request.contains_custom_tool_input() {\n        anyhow::bail!(\"llama.cpp does not support custom tools\");\n    }\n\n    let supports_tools = capabilities.supports_tools;\n    let supports_thinking = capabilities.supports_thinking;\n    let mut messages = Vec::new();\n\n    for message in request.messages {\n        let mut reasoning_content: Option<String> = None;\n        for content in message.content {\n            match content {\n                MessageContent::Text(text) => add_message_content_part(\n                    llama_cpp::MessagePart::Text { text },\n                    message.role,\n                    &mut messages,\n                    if supports_thinking && message.role == Role::Assistant {\n                        reasoning_content.take()\n                    } else {\n                        None","sourceCodeStart":682,"sourceCodeEnd":718,"githubUrl":"https://github.com/zed-industries/zed/blob/5a9b9558db01a6b906cec2fb70a797affdc58cdd/crates/language_models/src/provider/llama_cpp.rs#L682-L718","documentation":"build_llama_cpp_request bails when the incoming LanguageModelRequest contains custom tool input, because the llama.cpp chat-completion schema only models OpenAI-style function tools (name/description/parameters). This runs on every stream request, gated by the server's advertised LiveCapabilities.","triggerScenarios":"Connecting to a local llama.cpp server (OpenAI-compatible endpoint) and sending a request that includes a custom tool — e.g. an agent configured with a prompt-based tool while the model is served by llama.cpp.","commonSituations":"Local-model setups (Ollama-style workflows fronted by llama.cpp) reused with agents authored against cloud providers; switching providers in an existing thread that already carries custom tools.","solutions":["Re-create the tool as a function tool with a JSON Schema","Turn off the custom tool or the extension providing it for llama.cpp sessions","Use a function-tool-capable cloud provider for that agent","If your frontend supports it, declare the tool capability so the request is filtered earlier"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if capabilities.supports_tools && request.contains_custom_tool_input() {\n    anyhow::bail!(\"llama.cpp only supports function tools; strip custom tools before streaming\");\n}","typeGuard":"fn is_function_only_request(request: &LanguageModelRequest) -> bool {\n    !request.contains_custom_tool_input()\n}","tryCatchPattern":"match build_llama_cpp_request(model_name, supports_images, capabilities, request) {\n    Ok(req) => stream(req).await,\n    Err(e) if e.to_string().contains(\"custom tools\") => stream_without_custom_tools().await,\n    Err(e) => Err(e),\n}","preventionTips":["Local OpenAI-compatible servers generally accept function tools only — author tools accordingly","Filter tools by LiveCapabilities before building provider requests","Test agent configurations against every provider you plan to switch between"],"tags":["llama-cpp","llm","local-models","tools","capability-mismatch"],"backgroundTag":"llm-provider-unsupported-tool-type","analyzedSha":"5a9b9558db01a6b906cec2fb70a797affdc58cdd","analyzedAt":"2026-08-20T19:29:52.058Z","contentChangedAt":"2026-08-20T19:29:52.058Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}