{"record":{"id":"30edda6a9fef46f8","repo":"zylon-ai/private-gpt","slug":"initial-token-count-exceeds-token-limit","errorCode":null,"errorMessage":"Initial token count exceeds token limit","messagePattern":"Initial token count exceeds token limit","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/memory/trimming_memory.py","lineNumber":160,"sourceCode":"            start_on=start_on,\n            end_on=end_on,\n            tokenizer_fn=tokenizer_fn,\n            text_splitter=text_splitter or _default_text_splitter,\n            chat_store=chat_store or SimpleChatStore(),\n            chat_store_key=chat_store_key,\n        )\n\n    async def aget(\n        self, input: str | None = None, initial_token_count: int = 0, **kwargs: Any\n    ) -> list[ChatMessage]:\n        \"\"\"Get trimmed chat history based on configured strategy.\"\"\"\n        chat_history = await self.aget_all()\n\n        if not chat_history:\n            return []\n\n        if initial_token_count > self.token_limit:\n            raise ValueError(\"Initial token count exceeds token limit\")\n\n        max_tokens = self.token_limit - initial_token_count\n\n        if self.trim_strategy == TrimStrategy.FIRST:\n            return await self._trim_first_max_tokens(chat_history, max_tokens)\n        else:\n            return await self._trim_last_max_tokens(chat_history, max_tokens)\n\n    async def _trim_first_max_tokens(\n        self, messages: list[ChatMessage], max_tokens: int\n    ) -> list[ChatMessage]:\n        \"\"\"Keep the first messages up to the token limit.\"\"\"\n        if not messages:\n            return messages\n\n        # Find the maximum number of messages we can include\n        idx = 0\n        for i in range(len(messages)):","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/memory/trimming_memory.py#L142-L178","documentation":"Raised by TrimmingMemory.aget when initial_token_count already exceeds token_limit. aget subtracts initial_token_count from token_limit to get the remaining budget (max_tokens = token_limit - initial_token_count); a negative budget is nonsensical for either trim strategy, so the method fails fast instead of returning an inverted/negative trim result. initial_token_count typically represents tokens already consumed by the prompt/system parts elsewhere.","triggerScenarios":"Calling await memory.aget(input=..., initial_token_count=5000) with token_limit=4096; computing initial_token_count from the current prompt plus system message and exceeding the configured limit; token_limit set smaller than the fixed prompt overhead.","commonSituations":"Long system prompts eating most of a small token_limit; token_limit derived from a smaller model's context while prompts target a bigger one; counting the same tokens twice (once in initial count, once in history).","solutions":["Raise token_limit above your worst-case initial_token_count (or derive it from llm.metadata.context_window via from_defaults).","Recheck what initial_token_count includes — ensure it is only the fixed, non-history tokens.","Shorten the system prompt / static prefix that inflates initial_token_count.","Validate initial_token_count <= memory.token_limit before calling aget and degrade gracefully (empty history) if exceeded."],"exampleFix":"# before\nhistory = await memory.aget(initial_token_count=6000)  # token_limit=4096\n\n# after\nhistory = await memory.aget(initial_token_count=6000) if memory.token_limit >= 6000 else []","handlingStrategy":"validation","validationCode":"initial = count_prompt_tokens(system_prompt)  # fixed overhead only\nhistory = await memory.aget(initial_token_count=initial) if initial <= memory.token_limit else []","typeGuard":null,"tryCatchPattern":"try:\n    history = await memory.aget(initial_token_count=initial)\nexcept ValueError:\n    history = []  # no budget left; degrade to empty history","preventionTips":["Set token_limit with headroom above the largest system prompt you ship.","Count only fixed non-history tokens in initial_token_count; never double-count history."],"tags":["memory","validation","token-limit","context-window"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}