What causes "llama.cpp does not support custom tools" in zed-industries/zed?

Re-create the tool as a function tool with a JSON Schema

zed-industries/zed · error · anyhow::Error

llama.cpp does not support custom tools

Error message

llama.cpp does not support custom tools

What it means

build_llama_cpp_request bails when the incoming LanguageModelRequest contains custom tool input, because the llama.cpp chat-completion schema only models OpenAI-style function tools (name/description/parameters). This runs on every stream request, gated by the server's advertised LiveCapabilities.

Source

Thrown at crates/language_models/src/provider/llama_cpp.rs:700

                request,
                &extra_headers,
            )
            .await?;
            Ok(stream)
        });

        async move { Ok(future.await?.boxed()) }.boxed()
    }
}

fn build_llama_cpp_request(
    model_name: &str,
    supports_images: bool,
    capabilities: LiveCapabilities,
    request: LanguageModelRequest,
) -> Result<llama_cpp::ChatCompletionRequest> {
    if request.contains_custom_tool_input() {
        anyhow::bail!("llama.cpp does not support custom tools");
    }

    let supports_tools = capabilities.supports_tools;
    let supports_thinking = capabilities.supports_thinking;
    let mut messages = Vec::new();

    for message in request.messages {
        let mut reasoning_content: Option<String> = None;
        for content in message.content {
            match content {
                MessageContent::Text(text) => add_message_content_part(
                    llama_cpp::MessagePart::Text { text },
                    message.role,
                    &mut messages,
                    if supports_thinking && message.role == Role::Assistant {
                        reasoning_content.take()
                    } else {
                        None

View on GitHub (pinned to 5a9b9558db)

Solutions

Re-create the tool as a function tool with a JSON Schema
Turn off the custom tool or the extension providing it for llama.cpp sessions
Use a function-tool-capable cloud provider for that agent
If your frontend supports it, declare the tool capability so the request is filtered earlier

Defensive patterns

Strategy: validation

Validate before calling

if capabilities.supports_tools && request.contains_custom_tool_input() {
    anyhow::bail!("llama.cpp only supports function tools; strip custom tools before streaming");
}

Type guard

fn is_function_only_request(request: &LanguageModelRequest) -> bool {
    !request.contains_custom_tool_input()
}

Try / catch

match build_llama_cpp_request(model_name, supports_images, capabilities, request) {
    Ok(req) => stream(req).await,
    Err(e) if e.to_string().contains("custom tools") => stream_without_custom_tools().await,
    Err(e) => Err(e),
}

Prevention

Local OpenAI-compatible servers generally accept function tools only — author tools accordingly
Filter tools by LiveCapabilities before building provider requests
Test agent configurations against every provider you plan to switch between

When it happens

Trigger: Connecting to a local llama.cpp server (OpenAI-compatible endpoint) and sending a request that includes a custom tool — e.g. an agent configured with a prompt-based tool while the model is served by llama.cpp.

Common situations: Local-model setups (Ollama-style workflows fronted by llama.cpp) reused with agents authored against cloud providers; switching providers in an existing thread that already carries custom tools.

Related errors

AI-assisted analysis of zed-industries/zed@5a9b9558db (2026-08-20). Data as JSON: /api/errors/9455e56b779965f3. Report an issue: GitHub.