Mintplex-Labs/anything-llm · warning

[llm-proxy] Context exceeded — applying emergency compressio

Error message

[llm-proxy] Context exceeded — applying emergency compression and retrying

What it means

The llm-proxy forwarded a request upstream and got 400/413 whose body classified as CONTEXT_EXCEEDED — the conversation (messages plus tool outputs) is larger than the model's context window. The proxy reacts by running emergencyCompress (keeping the 6 most recent messages, truncating tool results to 300 chars) and retrying once; the warning marks that path.

Source

Thrown at open-computer/services/interface-service/llm-proxy/index.js:230

      }
    };

    try {
      activeLlmRequest = { startedAt: Date.now(), phase: "connecting" };
      broadcast({ type: "llm_status", state: "waiting" });
      const msgCount = req.body?.messages?.length || 0;
      broadcast({ type: "agent_log", content: `[llm-proxy] → waiting for response (${msgCount} msgs)` });

      let body = req.body;
      let { upstream, optimized } = await doRequest(body);

      // ── Context overflow: retry once with emergency compression ──────────
      if (!upstream.ok && (upstream.status === 400 || upstream.status === 413)) {
        const errPeek = await upstream.text();
        const classified = classifyLlmError(errPeek, upstream.status);

        if (classified.type === ERROR_TYPES.CONTEXT_EXCEEDED) {
          console.warn(`[llm-proxy] Context exceeded — applying emergency compression and retrying`);
          broadcast({ type: "agent_log", content: "[llm-proxy] Context exceeded — compressing history…" });

          const compressedMessages = emergencyCompress(body.messages || [], {
            keepRecent: 6,
            maxToolChars: 300,
          });
          body = { ...body, messages: compressedMessages };

          try {
            ({ upstream, optimized } = await doRequest(body, true));
          } catch (retryErr) {
            const retryClassified = classifyLlmError(retryErr.message);
            broadcastLlmError(retryClassified, retryErr.message);
            if (!res.headersSent)
              res.status(502).json({ error: `LLM proxy error: ${retryErr.message}` });
            return;
          }

View on GitHub (pinned to 3aec848f28)

Solutions

1. Switch to a model with a larger context window (32k+) for long agent sessions.
2. Start a new session/conversation to reset history when this recurs.
3. Reduce tool verbosity or the size of pasted content entering the prompt.
4. If it persists after compression, the retry error path will surface — treat that as the signal to change model or trim history.

// Rough preflight estimate before sending a long conversation:
const approxTokens = messages.reduce((n, m) => n + Math.ceil(JSON.stringify(m).length / 4), 0);
if (approxTokens > MODEL_CONTEXT * 0.8) {
  messages = emergencyCompress(messages, { keepRecent: 6, maxToolChars: 300 });
}

When it happens

Trigger: Long agent sessions where accumulated tool outputs exceed the model's window (common with 4k–8k context models); a huge file pasted into the conversation; verbose browser/DOM tool results accumulating over many turns.

Common situations: Small-context local models used for computer-use agents; marathon sessions without history trimming; a single enormous tool payload early in the conversation.

Related errors

• No LocalAi token context limit was set.
• Unknown provider: ${config.provider}. Please use a valid pro
• An error occurred while downloading the model
• HTTP ${resp.status}: ${(await resp.text()).slice(0, 200)}
• No token context limit was set.

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/bf3a1ba24c17d9e6. Report an issue: GitHub.