zylon-ai/private-gpt · error · Errors.RequestTooLarge

REQUEST_TOO_LARGE_USER_MSG

REQUEST_TOO_LARGE_USER_MSG

Error message

The message length {user_message_tokens} exceeds the maximum token limit {token_limit}.

What it means

Raised when the tokenized length of the last user message's text exceeds context.state.runtime.effective_token_limit. The interceptor uses the runtime tokenizer_fn via async_tokenizer and raises Errors.RequestTooLarge (code REQUEST_TOO_LARGE_USER_MSG) before the request reaches the LLM. This is a hard pre-flight guard against oversized prompts.

Source

Thrown at private_gpt/server/chat/interceptors/validator_request_interceptor.py:118

                    Errors.Codes.INVALID_REQUEST_AUDIO_SUPPORT_ERROR,
                )
            max_num_audios = max_audios_supported(llm, model_config)
            if len(audios) > max_num_audios:
                raise Errors.InvalidRequest(
                    f"The LLM supports a maximum of {max_num_audios} audios, but the message contains {len(audios)}",
                    Errors.Codes.INVALID_REQUEST_AUDIO_MAX_NUM_ERROR,
                )

        token_limit = context.state.runtime.effective_token_limit
        tokenize = context.state.runtime.tokenizer_fn
        if token_limit is None or tokenize is None:
            return

        user_message_tokens = len(
            await async_tokenizer(texts=user_text, tokenizer_fn=tokenize)
        )
        if user_message_tokens > token_limit:
            raise Errors.RequestTooLarge(
                f"The message length {user_message_tokens} exceeds the maximum token limit {token_limit}.",
                Errors.Codes.REQUEST_TOO_LARGE_USER_MSG,
            )

        # If a system message is present in the request messages, it's a misuse
        potential_system_message = self._system_message_text(
            context.state.input.request.messages
        )
        if potential_system_message:
            raise RuntimeError(
                "System messages should be as layer in the context stack."
            )

        # Prefer system prompt from the context stack, fall back to prompt
        system_prompt_block = (
            context.state.input.context_stack.to_system_prompt()
            or request.system.get_prompt()
            or None

View on GitHub (pinned to 4a030776a3)

Solutions

1. Trim or summarize the user message text so its token count is under effective_token_limit.
2. Raise effective_token_limit in settings/model config if the target model supports a larger context.
3. Move large content into attachments/context-stack layers instead of inline user text.
4. Pre-count tokens with the same runtime tokenizer_fn before sending and chunk the request.

Example fix

// before
await chat_facade.create_chat_event_generator(request=huge_text_request)  # 200k tokens vs 8k limit

// after
tokens = await async_tokenizer(texts=user_text, tokenizer_fn=tokenize)
if tokens and len(tokens) > effective_token_limit:
    user_text = user_text[: int(len(user_text) * effective_token_limit / len(tokens))]

user_text = ValidatorRequestInterceptor._extract_text(last_user_message)
tokens = len(await async_tokenizer(texts=user_text, tokenizer_fn=tokenize))
if token_limit is not None and tokens > token_limit:
    raise ValueError('trim before sending')

try:
    await chat_facade.create_chat_event_generator(request=request)
except Errors.RequestTooLarge:
    # truncate to a safe character estimate and retry once
    request.messages[-1] = shorten(request.messages[-1], token_limit)

When it happens

Trigger: Sending a user message whose extracted text (all TextBlocks joined) tokenizes to more tokens than effective_token_limit; e.g. pasting a huge document into the chat input when the effective limit is small (embedding/model limits, configured lngcs context, or reduced limits from context-stack layers).

Common situations: Long document pasted into chat; effective_token_limit lowered by model config or context-stack system prompt reservation; using a tokenizer_fn that does not match the model so counts are inflated; RAG context injected into the user message rather than the context stack.

Related errors

• INVALID_REQUEST_AUDIO_MAX_NUM_ERROR
• REQUEST_TOO_LARGE_SYSTEM_MSG
• The message length {user_message_tokens} and system prompt l
• Initial token count exceeds token limit
• Invalid page token

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/8336a91ce51de73f. Report an issue: GitHub.