zylon-ai/private-gpt · error · ValueError

The database schema is too long to fit in the model.

Error message

The database schema is too long to fit in the model.

What it means

Raised during SQL-generation prompt assembly when the token budget for the LLM is exhausted: available_tokens is computed as 80% of max_model_tokens minus the tokens already consumed by chat history and the database schema, and if that drops to zero or below the call is aborted. A code comment notes the planned TLDR/summarization strategy for schemas is not implemented, so oversize schemas are a hard failure.

Source

Thrown at private_gpt/components/tabular/database_query_generator.py:884

        final_history: list[ChatMessage] = [
            ChatMessage(
                role=MessageRole.SYSTEM,
                content=system_prompt,
            ),
            *messages,
        ]
        chat_history = await asyncio.to_thread(messages_to_history_str, final_history)
        available_tokens = max_model_tokens - (max_model_tokens // 5)  # 20% buffer
        if tokenizer is not None:
            current_tokens = len(tokenizer(chat_history))
            available_tokens -= current_tokens

        sampling_params: dict[str, Any] = {}
        if available_tokens <= 0:
            # TODO: TLDR strategy don't work well here,
            #  need to implement a especially TLDR for schema
            raise ValueError("The database schema is too long to fit in the model.")
        if available_tokens > 0:
            sampling_params["max_tokens"] = available_tokens

        response = await chat_service.chat(
            ResolvedChatRequest(
                messages=messages,
                system=ResolvedSystemConfig(
                    prompt=system_prompt, use_default_prompt=False
                ),
                condensation=CondensationConfig(enabled=False),
                sampling_params=sampling_params,
            )
        )

        # find the first text block in the response
        for block in response.content:
            if isinstance(block, TextBlock):
                raw_text = block.text

View on GitHub (pinned to 4a030776a3)

Solutions

1. Restrict the schema subset fed to the generator (select only relevant tables/views) so it fits the budget
2. Trim or shorten chat history before generation, or start a new conversation
3. Use a model with a larger context window / raise max_model_tokens if the deployment supports it

Example fix

# before: whole schema passed
schema_text = database_schema  # hundreds of tables -> ValueError
# after: relevant subset only
relevant = [t for t in database_schema.tables if t.name in wanted_tables]
schema_text = render_schema(relevant)

max_model_tokens = settings.llm.max_tokens  # as configured
available = max_model_tokens - (max_model_tokens // 5)
if tokenizer is not None:
    used = len(tokenizer(schema_text + history_text))
    if used >= available:
        raise ValueError(
            f"schema+history uses {used} tokens, budget is {available}; "
            "narrow the table selection"
        )

try:
    sql = await generator.generate(question)
except ValueError as e:
    if "too long" not in str(e):
        raise
    sql = await generator.generate(question, tables=relevant_subset)

When it happens

Trigger: Calling SQL generation against a database whose serialized schema (tables/columns/docs) plus chat history exceeds ~80% of the model's context window; a small max_model_tokens setting makes even modest schemas overflow.

Common situations: Pointing text-to-SQL at a huge ERP-style schema with hundreds of tables; long multi-turn conversations growing the history; misconfigured max_tokens for the model; models with small context windows.

Related errors

• Failed to generate SQL query
• Generated SQL query is invalid: {error_str}
• REQUEST_TOO_LARGE_USER_MSG
• REQUEST_TOO_LARGE_SYSTEM_MSG
• The message length {user_message_tokens} and system prompt l

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/85759ea1afa32302. Report an issue: GitHub.