{"record":{"id":"d98d8dab31535ea6","repo":"sigoden/aichat","slug":"exceed-max-input-tokens-limit","errorCode":null,"errorMessage":"Exceed max_input_tokens limit","messagePattern":"Exceed max_input_tokens limit","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/client/model.rs","lineNumber":288,"sourceCode":"\n    pub fn total_tokens(&self, messages: &[Message]) -> usize {\n        if messages.is_empty() {\n            return 0;\n        }\n        let num_messages = messages.len();\n        let message_tokens = self.messages_tokens(messages);\n        if messages[num_messages - 1].role.is_user() {\n            num_messages * PER_MESSAGES_TOKENS + message_tokens\n        } else {\n            (num_messages - 1) * PER_MESSAGES_TOKENS + message_tokens\n        }\n    }\n\n    pub fn guard_max_input_tokens(&self, messages: &[Message]) -> Result<()> {\n        let total_tokens = self.total_tokens(messages) + BASIS_TOKENS;\n        if let Some(max_input_tokens) = self.data.max_input_tokens {\n            if total_tokens >= max_input_tokens {\n                bail!(\"Exceed max_input_tokens limit\")\n            }\n        }\n        Ok(())\n    }\n}\n\n#[derive(Debug, Clone, Default, Serialize, Deserialize)]\npub struct ModelData {\n    pub name: String,\n    #[serde(default = \"default_model_type\", rename = \"type\")]\n    pub model_type: String,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub real_name: Option<String>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub max_input_tokens: Option<usize>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub input_price: Option<f64>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]","sourceCodeStart":270,"sourceCodeEnd":306,"githubUrl":"https://github.com/sigoden/aichat/blob/82976d349ad97ac9aae0655ad631dace5e2a6385/src/client/model.rs#L270-L306","documentation":"`guard_max_input_tokens` (src/client/model.rs:288) bails when the estimated total token count of the message list (plus BASIS_TOKENS overhead) meets or exceeds the client's configured max_input_tokens. The library refuses the request up front rather than letting the provider reject an oversized prompt.","triggerScenarios":"prepare_completion_data (via guard_max_input_tokens) called with messages whose total_tokens + BASIS_TOKENS >= data.max_input_tokens — e.g. very long conversations, pasted large documents, or a config where max_input_tokens is set below actual usage.","commonSituations":"max_input_tokens misconfigured too low for the local model's real context (common with Ollama/local models); RAG/summarization flows accumulating huge context; long chat sessions never trimmed.","solutions":["Increase max_input_tokens in the client config if the provider/model truly supports a larger context window.","Trim or summarize older messages in the conversation before the call.","Split large documents into smaller chunks and send fewer per request.","Verify total_tokens estimation isn't inflated (tokenizer mismatch) and adjust BASIS_TOKENS/config accordingly."],"exampleFix":"// before: config caps below usage\n\"max_input_tokens\": 4096  // sending ~8k tokens\n// after\n\"max_input_tokens\": 32768","handlingStrategy":"validation","validationCode":"// estimate tokens before sending and trim if needed\nlet est = total_tokens(&messages) + BASIS_TOKENS;\nlet messages = if est >= max_input_tokens { trim_or_summarize(messages, max_input_tokens) } else { messages };","typeGuard":"fn fits_context(messages: &[Message], max_input_tokens: usize) -> bool {\n    total_tokens(messages) + BASIS_TOKENS < max_input_tokens\n}","tryCatchPattern":"match client.chat_completions(&req).await {\n    Err(e) if e.to_string().contains(\"Exceed max_input_tokens\") => {\n        let mut req = req.clone();\n        req.messages = trim_oldest(&req.messages);\n        client.chat_completions(&req).await\n    }\n    other => other,\n}","preventionTips":["Set max_input_tokens to the model's real context window (especially for local models).","Trim or summarize long conversation histories before each call.","Chunk large documents instead of pasting them whole.","Check token estimates before sending rather than after failure."],"tags":["token-limit","payload-too-large","rust","llm"],"backgroundTag":"payload-too-large","analyzedSha":"82976d349ad97ac9aae0655ad631dace5e2a6385","analyzedAt":"2026-09-09T18:33:06.139Z","contentChangedAt":"2026-09-09T18:33:06.139Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}