zed-industries/zed · error

Failed to connect to llama.cpp API: {} {}

Error message

Failed to connect to llama.cpp API: {} {}

What it means

The llama.cpp streaming completion HTTP request returned a non-success status; the status and response body are embedded in this error, meaning the local llama.cpp server refused or failed the completion request.

Source

Thrown at crates/llama_cpp/src/llama_cpp.rs:487

                            None
                        } else {
                            match serde_json::from_str::<ResponseStreamResult>(line) {
                                Ok(ResponseStreamResult::Ok(response)) => Some(Ok(response)),
                                Ok(ResponseStreamResult::Err { error }) => {
                                    Some(Err(anyhow!(error.message)))
                                }
                                Err(error) => Some(Err(anyhow!(error))),
                            }
                        }
                    }
                    Err(error) => Some(Err(anyhow!(error))),
                }
            })
            .boxed())
    } else {
        let mut body = String::new();
        response.body_mut().read_to_string(&mut body).await?;
        anyhow::bail!(
            "Failed to connect to llama.cpp API: {} {}",
            response.status(),
            body,
        );
    }
}

/// Lists the models the server is serving via `GET /v1/models`.
pub async fn get_models(
    client: &dyn HttpClient,
    api_url: &str,
    api_key: Option<&str>,
    extra_headers: &CustomHeaders,
) -> Result<Vec<ModelEntry>> {
    let uri = format!("{api_url}/v1/models");
    let request = HttpRequest::builder()
        .method(Method::GET)
        .uri(uri)

View on GitHub (pinned to 5a9b9558db)

Solutions

1. Verify the llama.cpp server is running and the API URL/port is correct
2. Check that the model is loaded on the server
3. Retry if the server was temporarily overloaded or restarting
4. Inspect the logged status code and body for the server-side error

When it happens

Trigger: Thrown at crates/llama_cpp/src/llama_cpp.rs:541 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of zed-industries/zed@5a9b9558db (2026-08-20). Data as JSON: /api/errors/43f87e163d99a141. Report an issue: GitHub.