sigoden/aichat · error

Exceed max_input_tokens limit

Error message

Exceed max_input_tokens limit

What it means

`guard_max_input_tokens` (src/client/model.rs:288) bails when the estimated total token count of the message list (plus BASIS_TOKENS overhead) meets or exceeds the client's configured max_input_tokens. The library refuses the request up front rather than letting the provider reject an oversized prompt.

Solutions

1. Increase max_input_tokens in the client config if the provider/model truly supports a larger context window.
2. Trim or summarize older messages in the conversation before the call.
3. Split large documents into smaller chunks and send fewer per request.
4. Verify total_tokens estimation isn't inflated (tokenizer mismatch) and adjust BASIS_TOKENS/config accordingly.

Example fix

// before: config caps below usage
"max_input_tokens": 4096  // sending ~8k tokens
// after
"max_input_tokens": 32768

// estimate tokens before sending and trim if needed
let est = total_tokens(&messages) + BASIS_TOKENS;
let messages = if est >= max_input_tokens { trim_or_summarize(messages, max_input_tokens) } else { messages };

fn fits_context(messages: &[Message], max_input_tokens: usize) -> bool {
    total_tokens(messages) + BASIS_TOKENS < max_input_tokens
}

match client.chat_completions(&req).await {
    Err(e) if e.to_string().contains("Exceed max_input_tokens") => {
        let mut req = req.clone();
        req.messages = trim_oldest(&req.messages);
        client.chat_completions(&req).await
    }
    other => other,
}

When it happens

Trigger: prepare_completion_data (via guard_max_input_tokens) called with messages whose total_tokens + BASIS_TOKENS >= data.max_input_tokens — e.g. very long conversations, pasted large documents, or a config where max_input_tokens is set below actual usage.

Common situations: max_input_tokens misconfigured too low for the local model's real context (common with Ollama/local models); RAG/summarization flows accumulating huge context; long chat sessions never trimmed.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• {error}
• {message}
• Invalid response data
• No models
• Unknown client

AI-assisted analysis of sigoden/aichat@82976d349a (2026-09-09). Data as JSON: /api/errors/d98d8dab31535ea6. Report an issue: GitHub.

Appendix: source

Thrown at src/client/model.rs:288

    pub fn total_tokens(&self, messages: &[Message]) -> usize {
        if messages.is_empty() {
            return 0;
        }
        let num_messages = messages.len();
        let message_tokens = self.messages_tokens(messages);
        if messages[num_messages - 1].role.is_user() {
            num_messages * PER_MESSAGES_TOKENS + message_tokens
        } else {
            (num_messages - 1) * PER_MESSAGES_TOKENS + message_tokens
        }
    }

    pub fn guard_max_input_tokens(&self, messages: &[Message]) -> Result<()> {
        let total_tokens = self.total_tokens(messages) + BASIS_TOKENS;
        if let Some(max_input_tokens) = self.data.max_input_tokens {
            if total_tokens >= max_input_tokens {
                bail!("Exceed max_input_tokens limit")
            }
        }
        Ok(())
    }
}

#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct ModelData {
    pub name: String,
    #[serde(default = "default_model_type", rename = "type")]
    pub model_type: String,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub real_name: Option<String>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub max_input_tokens: Option<usize>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub input_price: Option<f64>,
    #[serde(skip_serializing_if = "Option::is_none")]

View on GitHub (pinned to 82976d349a)