zylon-ai/private-gpt · error · Errors.RequestTooLarge

The message length {user_message_tokens} and system prompt l

Error message

The message length {user_message_tokens} and system prompt length {system_tokens} exceed the maximum token limit {token_limit}.

What it means

Raised when user_message_tokens + system_tokens together exceed effective_token_limit, even though each individually passes (the per-checks at REQUEST_TOO_LARGE_USER_MSG and REQUEST_TOO_LARGE_SYSTEM_MSG did not fire). Raised as Errors.RequestTooLarge without an explicit code. It is the combined-budget guard run last in the interceptor.

Source

Thrown at private_gpt/server/chat/interceptors/validator_request_interceptor.py:160

                else []
            )
            if system_prompt_block
            else None
        )
        system_tokens = (
            len(await async_tokenizer(texts=system_prompt, tokenizer_fn=tokenize))
            if system_prompt
            else 0
        )
        if system_tokens > token_limit:
            raise Errors.RequestTooLarge(
                f"The system prompt length {system_tokens} exceeds the maximum token limit {token_limit}.",
                Errors.Codes.REQUEST_TOO_LARGE_SYSTEM_MSG,
            )

        combined = user_message_tokens + system_tokens
        if combined > token_limit:
            raise Errors.RequestTooLarge(
                f"The message length {user_message_tokens} and system prompt length {system_tokens} "
                f"exceed the maximum token limit {token_limit}."
            )

        return

    @staticmethod
    def _extract_text(message: ChatMessage) -> str:
        """Extract normalized text from message blocks."""
        parts = [
            block.text.strip()
            for block in message.blocks
            if isinstance(block, TextBlock) and block.text and block.text.strip()
        ]
        return "\n".join(parts)

    @staticmethod
    def _last_user_message(messages: list[ChatMessage]) -> ChatMessage:

View on GitHub (pinned to 4a030776a3)

Solutions

1. Trim the user message, the system prompt, or both so their combined token count is under effective_token_limit.
2. Raise effective_token_limit if the underlying model supports it.
3. Reserve a token budget for the system prompt and validate user text against (token_limit - system_tokens) before sending.
4. Compact or prune context-stack layers to free budget.

Example fix

# before
budget = effective_token_limit  # validated user text alone against full limit

# after
system_tokens = len(await async_tokenizer(texts=system_prompt, tokenizer_fn=tokenize)) if system_prompt else 0
budget = effective_token_limit - system_tokens  # validate user text against remaining budget

budget = effective_token_limit - system_tokens
user_tokens = len(await async_tokenizer(texts=user_text, tokenizer_fn=tokenize))
if user_tokens > budget:
    user_text = user_text[:int(len(user_text) * budget / user_tokens)]

try:
    await chat_facade.create_chat_event_generator(request=request)
except Errors.RequestTooLarge:
    # reduce user text to the remaining budget and retry once

When it happens

Trigger: A moderately long user message plus a moderately long system prompt whose sum crosses the limit; e.g. 5k-token system prompt + 4k-token user message against an 8k effective limit.

Common situations: Growing system prompts (context layers) leaving shrinking headroom for user input across a session; per-component validation client-side that forgets to sum both parts.

Related errors

• REQUEST_TOO_LARGE_SYSTEM_MSG
• REQUEST_TOO_LARGE_USER_MSG
• Initial token count exceeds token limit
• Invalid page token
• The database schema is too long to fit in the model.

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/1e4114e19a228341. Report an issue: GitHub.