SillyTavern/SillyTavern · error · Error

LlamaCpp: Failed to get vector for text: ${response.statusTe

Error message

LlamaCpp: Failed to get vector for text: ${response.statusText} ${responseText}

What it means

The LlamaCpp vectors connector POSTs { input: texts } to a LlamaCpp endpoint and throws a descriptive error including response.statusText and the response text when response.ok is false. LlamaCpp is a self-hosted local inference server, so failure usually means the server rejected the embedding request or is unreachable in a degraded way.

Source

Thrown at src/vectors/llamacpp-vectors.js:31

 */
export async function getLlamaCppBatchVector(texts, apiUrl, directories) {
    const url = new URL(urlJoin(trimV1(apiUrl), '/v1/embeddings'));

    const headers = {};
    setAdditionalHeadersByType(headers, TEXTGEN_TYPES.LLAMACPP, apiUrl, directories);

    const response = await fetch(url, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            ...headers,
        },
        body: JSON.stringify({ input: texts }),
    });

    if (!response.ok) {
        const responseText = await response.text();
        throw new Error(`LlamaCpp: Failed to get vector for text: ${response.statusText} ${responseText}`);
    }

    /** @type {any} */
    const data = await response.json();

    if (!Array.isArray(data?.data)) {
        throw new Error('API response was not an array');
    }

    // Sort data by x.index to ensure the order is correct
    data.data.sort((a, b) => a.index - b.index);

    const vectors = data.data.map(x => x.embedding);
    return vectors;
}

/**
 * Gets the vector for the given text from LlamaCpp

View on GitHub (pinned to 8172dcd0ee)

Solutions

1. Read the embedded statusText and responseText to see LlamaCpp's error message.
2. Restart LlamaCpp with embedding support enabled for the loaded model.
3. Use a model that supports embeddings.
4. Reduce input text length to fit the model's context window.
5. Confirm the API URL points at the correct LlamaCpp embedding endpoint.

// Validate the LlamaCpp URL and inputs before calling
if (!url) throw new Error('LlamaCpp URL is not configured');
if (!Array.isArray(texts) || texts.length === 0) throw new Error('texts must be a non-empty array');

try {
    return await getLlamaCppBatchVector(texts, ...);
} catch (e) {
    if (e.message.startsWith('LlamaCpp: Failed to get vector')) {
        // inspect embedded statusText/responseText; retry once for transient server errors
    } else throw e;
}

When it happens

Trigger: LlamaCpp returns non-2xx: model not loaded, embedding not supported by the loaded model, input too long, server misconfigured, or wrong endpoint path.

Common situations: LlamaCpp server started without --embedding support, model that doesn't produce embeddings, context length exceeded by input, wrong API URL, or LlamaCpp version with a different endpoint.

Related errors

• Extras request failed
• Ollama: Failed to get batch vectors: ${response.statusText}
• API request failed
• ${apiName} batch request failed
• API response was not an array

AI-assisted analysis of SillyTavern/SillyTavern@8172dcd0ee (2026-08-13). Data as JSON: /api/errors/0104c868cb305bb3. Report an issue: GitHub.