zylon-ai/private-gpt · error · Errors.RequestTooLarge

REQUEST_TOO_LARGE_SYSTEM_MSG

REQUEST_TOO_LARGE_SYSTEM_MSG

Error message

The system prompt length {system_tokens} exceeds the maximum token limit {token_limit}.

What it means

Raised when the resolved system prompt (context-stack system prompt joined, falling back to request.system.get_prompt()) alone tokenizes to more than effective_token_limit. Errors.RequestTooLarge with code REQUEST_TOO_LARGE_SYSTEM_MSG. Even a valid user message cannot coexist with a system prompt that alone blows the budget.

Source

Thrown at private_gpt/server/chat/interceptors/validator_request_interceptor.py:153

            or request.system.get_prompt()
            or None
        )
        system_prompt = (
            "\n".join(
                [block.text for block in system_prompt_block]
                if system_prompt_block
                else []
            )
            if system_prompt_block
            else None
        )
        system_tokens = (
            len(await async_tokenizer(texts=system_prompt, tokenizer_fn=tokenize))
            if system_prompt
            else 0
        )
        if system_tokens > token_limit:
            raise Errors.RequestTooLarge(
                f"The system prompt length {system_tokens} exceeds the maximum token limit {token_limit}.",
                Errors.Codes.REQUEST_TOO_LARGE_SYSTEM_MSG,
            )

        combined = user_message_tokens + system_tokens
        if combined > token_limit:
            raise Errors.RequestTooLarge(
                f"The message length {user_message_tokens} and system prompt length {system_tokens} "
                f"exceed the maximum token limit {token_limit}."
            )

        return

    @staticmethod
    def _extract_text(message: ChatMessage) -> str:
        """Extract normalized text from message blocks."""
        parts = [
            block.text.strip()

View on GitHub (pinned to 4a030776a3)

Solutions

1. Shorten the system prompt or reduce context-stack system layers until system_tokens <= effective_token_limit.
2. Raise effective_token_limit in the model/settings configuration.
3. Move bulky reference content out of the system prompt into retrieval (RAG) or attachments.
4. Audit context_stack.to_system_prompt() output length at request time and compact layers.

Example fix

# before
context_stack.add_system_layer(huge_prompt_50k_tokens)  # limit 8k

# after
context_stack.add_system_layer(compact_instructions_2k_tokens)

system_prompt = context_stack.to_system_prompt() or request.system.get_prompt()
system_tokens = len(await async_tokenizer(texts=system_prompt, tokenizer_fn=tokenize)) if system_prompt else 0
assert system_tokens <= effective_token_limit

try:
    await chat_facade.create_chat_event_generator(request=request)
except Errors.RequestTooLarge:
    context_stack.compact_system_layers()  # shrink and retry

When it happens

Trigger: Configuring a very large system prompt (long instructions, injected documents as a context-stack layer) while effective_token_limit is small; stacking multiple context layers whose combined system prompt exceeds the limit.

Common situations: RAG content placed in the system layer; accumulating context-stack layers across a session until the prompt grows past the limit; lowering the model/context settings without shrinking the stored system prompt.

Related errors

• The message length {user_message_tokens} and system prompt l
• REQUEST_TOO_LARGE_USER_MSG
• Initial token count exceeds token limit
• Invalid page token
• The database schema is too long to fit in the model.

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/3c47b9285540a8d5. Report an issue: GitHub.