{"record":{"id":"85759ea1afa32302","repo":"zylon-ai/private-gpt","slug":"the-database-schema-is-too-long-to-fit-in-the-mode","errorCode":null,"errorMessage":"The database schema is too long to fit in the model.","messagePattern":"The database schema is too long to fit in the model\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/tabular/database_query_generator.py","lineNumber":884,"sourceCode":"\n        final_history: list[ChatMessage] = [\n            ChatMessage(\n                role=MessageRole.SYSTEM,\n                content=system_prompt,\n            ),\n            *messages,\n        ]\n        chat_history = await asyncio.to_thread(messages_to_history_str, final_history)\n        available_tokens = max_model_tokens - (max_model_tokens // 5)  # 20% buffer\n        if tokenizer is not None:\n            current_tokens = len(tokenizer(chat_history))\n            available_tokens -= current_tokens\n\n        sampling_params: dict[str, Any] = {}\n        if available_tokens <= 0:\n            # TODO: TLDR strategy don't work well here,\n            #  need to implement a especially TLDR for schema\n            raise ValueError(\"The database schema is too long to fit in the model.\")\n        if available_tokens > 0:\n            sampling_params[\"max_tokens\"] = available_tokens\n\n        response = await chat_service.chat(\n            ResolvedChatRequest(\n                messages=messages,\n                system=ResolvedSystemConfig(\n                    prompt=system_prompt, use_default_prompt=False\n                ),\n                condensation=CondensationConfig(enabled=False),\n                sampling_params=sampling_params,\n            )\n        )\n\n        # find the first text block in the response\n        for block in response.content:\n            if isinstance(block, TextBlock):\n                raw_text = block.text","sourceCodeStart":866,"sourceCodeEnd":902,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/tabular/database_query_generator.py#L866-L902","documentation":"Raised during SQL-generation prompt assembly when the token budget for the LLM is exhausted: available_tokens is computed as 80% of max_model_tokens minus the tokens already consumed by chat history and the database schema, and if that drops to zero or below the call is aborted. A code comment notes the planned TLDR/summarization strategy for schemas is not implemented, so oversize schemas are a hard failure.","triggerScenarios":"Calling SQL generation against a database whose serialized schema (tables/columns/docs) plus chat history exceeds ~80% of the model's context window; a small max_model_tokens setting makes even modest schemas overflow.","commonSituations":"Pointing text-to-SQL at a huge ERP-style schema with hundreds of tables; long multi-turn conversations growing the history; misconfigured max_tokens for the model; models with small context windows.","solutions":["Restrict the schema subset fed to the generator (select only relevant tables/views) so it fits the budget","Trim or shorten chat history before generation, or start a new conversation","Use a model with a larger context window / raise max_model_tokens if the deployment supports it"],"exampleFix":"# before: whole schema passed\nschema_text = database_schema  # hundreds of tables -> ValueError\n# after: relevant subset only\nrelevant = [t for t in database_schema.tables if t.name in wanted_tables]\nschema_text = render_schema(relevant)","handlingStrategy":"validation","validationCode":"max_model_tokens = settings.llm.max_tokens  # as configured\navailable = max_model_tokens - (max_model_tokens // 5)\nif tokenizer is not None:\n    used = len(tokenizer(schema_text + history_text))\n    if used >= available:\n        raise ValueError(\n            f\"schema+history uses {used} tokens, budget is {available}; \"\n            \"narrow the table selection\"\n        )","typeGuard":null,"tryCatchPattern":"try:\n    sql = await generator.generate(question)\nexcept ValueError as e:\n    if \"too long\" not in str(e):\n        raise\n    sql = await generator.generate(question, tables=relevant_subset)","preventionTips":["Feed only the relevant table subset instead of the full schema","Monitor schema token count as it grows; alert before it nears the 80% budget"],"tags":["database","text-to-sql","context-window","tokens","llm"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}