run-llama/llama_index · error · ValueError

Initial token count exceeds token limit

Error message

Initial token count exceeds token limit

What it means

ChatMemoryBuffer.get() accepts initial_token_count to reserve space for tokens outside the memory (typically the current user query plus prompt overhead). If that reservation alone already exceeds token_limit, no room remains for any history and the request is rejected with ValueError instead of returning an arbitrarily truncated history.

Source

Thrown at llama-index-core/llama_index/core/memory/chat_memory_buffer.py:121

        if "chat_history" in data:
            chat_history = data.pop("chat_history")
            simple_store = SimpleChatStore(store={DEFAULT_CHAT_STORE_KEY: chat_history})
            data["chat_store"] = simple_store
        elif "chat_store" in data:
            chat_store_dict = data.pop("chat_store")
            chat_store = load_chat_store(chat_store_dict)
            data["chat_store"] = chat_store

        return cls(**data)

    def get(
        self, input: Optional[str] = None, initial_token_count: int = 0, **kwargs: Any
    ) -> List[ChatMessage]:
        """Get chat history."""
        chat_history = self.get_all()

        if initial_token_count > self.token_limit:
            raise ValueError("Initial token count exceeds token limit")

        message_count = len(chat_history)

        cur_messages = chat_history[-message_count:]
        token_count = self._token_count_for_messages(cur_messages) + initial_token_count

        while token_count > self.token_limit and message_count > 1:
            message_count -= 1
            while message_count > 1 and chat_history[-message_count].role in (
                MessageRole.TOOL,
                MessageRole.ASSISTANT,
            ):
                # we cannot have an assistant message at the start of the chat history
                # if after removal of the first, we have an assistant message,
                # we need to remove the assistant message too
                #
                # all tool messages should be preceded by an assistant message
                # if we remove a tool message, we need to remove the assistant message too

View on GitHub (pinned to afd0fef371)

Solutions

1. Increase token_limit: ChatMemoryBuffer.from_defaults(llm=llm, token_limit=bigger_value).
2. Reduce/truncate the initial token payload (shorten the query or pre-summarize it) before calling get().
3. Use an LLM with a larger context window so the derived limit accommodates the query.

Example fix

# before
memory = ChatMemoryBuffer(token_limit=512)
history = memory.get(initial_token_count=800)  # ValueError
# after
memory = ChatMemoryBuffer(token_limit=4000)
history = memory.get(initial_token_count=800)

def can_fit_initial_count(memory, initial_token_count: int) -> bool:
    return 0 <= initial_token_count <= memory.token_limit

if not can_fit_initial_count(memory, initial_token_count):
    initial_token_count = memory.token_limit - 1  # or raise with context

def is_within_token_limit(memory, count: int) -> bool:
    return count < memory.token_limit

try:
    history = memory.get(initial_token_count=n)
except ValueError as e:
    if "Initial token count exceeds" in str(e):
        history = memory.get(initial_token_count=0)  # degrade gracefully
    else:
        raise

When it happens

Trigger: Calling memory.get(initial_token_count=N) with N greater than the buffer's token_limit — e.g. a very large query token count combined with a small buffer (default DEFAULT_TOKEN_LIMIT or context_window*ratio derived limit).

Common situations: Long user queries pasted into chatbots with small memory limits; embedding-heavy initial counts; a small context-window LLM (small derived token_limit) receiving a moderately large prompt.

Related errors

• Token limit must be set and greater than 0.
• Unexpected kwargs: {kwargs}
• Token limit for full-text messages must be set and greater t
• Unexpected keyword arguments: {kwargs}
• Max iterations of {max_iterations} reached! Either something

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/84c5e59f354bee6b. Report an issue: GitHub.