{"record":{"id":"84c5e59f354bee6b","repo":"run-llama/llama_index","slug":"initial-token-count-exceeds-token-limit","errorCode":null,"errorMessage":"Initial token count exceeds token limit","messagePattern":"Initial token count exceeds token limit","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/memory/chat_memory_buffer.py","lineNumber":121,"sourceCode":"        if \"chat_history\" in data:\n            chat_history = data.pop(\"chat_history\")\n            simple_store = SimpleChatStore(store={DEFAULT_CHAT_STORE_KEY: chat_history})\n            data[\"chat_store\"] = simple_store\n        elif \"chat_store\" in data:\n            chat_store_dict = data.pop(\"chat_store\")\n            chat_store = load_chat_store(chat_store_dict)\n            data[\"chat_store\"] = chat_store\n\n        return cls(**data)\n\n    def get(\n        self, input: Optional[str] = None, initial_token_count: int = 0, **kwargs: Any\n    ) -> List[ChatMessage]:\n        \"\"\"Get chat history.\"\"\"\n        chat_history = self.get_all()\n\n        if initial_token_count > self.token_limit:\n            raise ValueError(\"Initial token count exceeds token limit\")\n\n        message_count = len(chat_history)\n\n        cur_messages = chat_history[-message_count:]\n        token_count = self._token_count_for_messages(cur_messages) + initial_token_count\n\n        while token_count > self.token_limit and message_count > 1:\n            message_count -= 1\n            while message_count > 1 and chat_history[-message_count].role in (\n                MessageRole.TOOL,\n                MessageRole.ASSISTANT,\n            ):\n                # we cannot have an assistant message at the start of the chat history\n                # if after removal of the first, we have an assistant message,\n                # we need to remove the assistant message too\n                #\n                # all tool messages should be preceded by an assistant message\n                # if we remove a tool message, we need to remove the assistant message too","sourceCodeStart":103,"sourceCodeEnd":139,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/memory/chat_memory_buffer.py#L103-L139","documentation":"ChatMemoryBuffer.get() accepts initial_token_count to reserve space for tokens outside the memory (typically the current user query plus prompt overhead). If that reservation alone already exceeds token_limit, no room remains for any history and the request is rejected with ValueError instead of returning an arbitrarily truncated history.","triggerScenarios":"Calling memory.get(initial_token_count=N) with N greater than the buffer's token_limit — e.g. a very large query token count combined with a small buffer (default DEFAULT_TOKEN_LIMIT or context_window*ratio derived limit).","commonSituations":"Long user queries pasted into chatbots with small memory limits; embedding-heavy initial counts; a small context-window LLM (small derived token_limit) receiving a moderately large prompt.","solutions":["Increase token_limit: ChatMemoryBuffer.from_defaults(llm=llm, token_limit=bigger_value).","Reduce/truncate the initial token payload (shorten the query or pre-summarize it) before calling get().","Use an LLM with a larger context window so the derived limit accommodates the query."],"exampleFix":"# before\nmemory = ChatMemoryBuffer(token_limit=512)\nhistory = memory.get(initial_token_count=800)  # ValueError\n# after\nmemory = ChatMemoryBuffer(token_limit=4000)\nhistory = memory.get(initial_token_count=800)","handlingStrategy":"validation","validationCode":"def can_fit_initial_count(memory, initial_token_count: int) -> bool:\n    return 0 <= initial_token_count <= memory.token_limit\n\nif not can_fit_initial_count(memory, initial_token_count):\n    initial_token_count = memory.token_limit - 1  # or raise with context","typeGuard":"def is_within_token_limit(memory, count: int) -> bool:\n    return count < memory.token_limit","tryCatchPattern":"try:\n    history = memory.get(initial_token_count=n)\nexcept ValueError as e:\n    if \"Initial token count exceeds\" in str(e):\n        history = memory.get(initial_token_count=0)  # degrade gracefully\n    else:\n        raise","preventionTips":["Size token_limit generously relative to the largest expected query.","Compute and clamp initial_token_count before calling get().","Derive token_limit from the actual LLM context window via from_defaults(llm=...)."],"tags":["llama-index","memory","token-limit","chat-history","validation"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}