{"record":{"id":"f176bdebf39d36cf","repo":"run-llama/llama_index","slug":"token-limit-for-full-text-messages-must-be-set-and","errorCode":null,"errorMessage":"Token limit for full-text messages must be set and greater than 0.","messagePattern":"Token limit for full-text messages must be set and greater than 0\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/memory/chat_summary_memory_buffer.py","lineNumber":72,"sourceCode":"    chat_store: SerializeAsAny[BaseChatStore] = Field(default_factory=SimpleChatStore)\r\n    chat_store_key: str = Field(default=DEFAULT_CHAT_STORE_KEY)\r\n\r\n    _token_count: int = PrivateAttr(default=0)\r\n\r\n    @field_serializer(\"chat_store\")\r\n    def serialize_courses_in_order(self, chat_store: BaseChatStore) -> dict:\r\n        res = chat_store.model_dump()\r\n        res.update({\"class_name\": chat_store.class_name()})\r\n        return res\r\n\r\n    @model_validator(mode=\"before\")\r\n    @classmethod\r\n    def validate_memory(cls, values: dict) -> dict:\r\n        \"\"\"Validate the memory.\"\"\"\r\n        # Validate token limits\r\n        token_limit = values.get(\"token_limit\", -1)\r\n        if token_limit < 1:\r\n            raise ValueError(\r\n                \"Token limit for full-text messages must be set and greater than 0.\"\r\n            )\r\n\r\n        # Validate tokenizer -- this avoids errors when loading from json/dict\r\n        tokenizer_fn = values.get(\"tokenizer_fn\")\r\n        if tokenizer_fn is None:\r\n            values[\"tokenizer_fn\"] = get_tokenizer()\r\n\r\n        return values\r\n\r\n    @classmethod\r\n    def from_defaults(\r\n        cls,\r\n        chat_history: Optional[List[ChatMessage]] = None,\r\n        llm: Optional[LLM] = None,\r\n        chat_store: Optional[BaseChatStore] = None,\r\n        chat_store_key: str = DEFAULT_CHAT_STORE_KEY,\r\n        token_limit: Optional[int] = None,\r","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/memory/chat_summary_memory_buffer.py#L54-L90","documentation":"ChatSummaryMemoryBuffer (which summarizes old messages instead of dropping them) uses the same pydantic validation pattern as ChatMemoryBuffer: token_limit must be present and >= 1, otherwise ValueError. The limit governs when full-text messages get summarized into a compressed summary.","triggerScenarios":"Constructing ChatSummaryMemoryBuffer(...) directly without token_limit, with 0/negative, or deserializing a dict lacking the key — note it also requires an llm for summarization.","commonSituations":"Switching from ChatMemoryBuffer and assuming defaults exist on the raw constructor; loading persisted memory JSON that omitted token_limit; passing None explicitly.","solutions":["Use ChatSummaryMemoryBuffer.from_defaults(llm=llm, ...) which derives the limit from the LLM context window.","Pass token_limit explicitly as a positive integer.","Verify persisted dicts include token_limit before reloading."],"exampleFix":"# before\nmemory = ChatSummaryMemoryBuffer(llm=llm)  # ValueError: no token_limit\n# after\nmemory = ChatSummaryMemoryBuffer.from_defaults(llm=llm)\n# or\nmemory = ChatSummaryMemoryBuffer(llm=llm, token_limit=3000)","handlingStrategy":"validation","validationCode":"def is_valid_summary_memory_config(data: dict) -> bool:\n    tl = data.get(\"token_limit\", -1)\n    return isinstance(tl, int) and tl >= 1 and data.get(\"llm\") is not None\n\nassert is_valid_summary_memory_config(config), \"token_limit >= 1 and llm are required\"","typeGuard":"def has_valid_token_limit(memory_dict: dict) -> bool:\n    tl = memory_dict.get(\"token_limit\", -1)\n    return isinstance(tl, int) and tl >= 1","tryCatchPattern":"try:\n    memory = ChatSummaryMemoryBuffer(llm=llm)\nexcept ValueError as e:\n    if \"Token limit for full-text\" in str(e):\n        memory = ChatSummaryMemoryBuffer(llm=llm, token_limit=3000)\n    else:\n        raise","preventionTips":["Use from_defaults(llm=...) for ChatSummaryMemoryBuffer to auto-derive the limit.","Include token_limit whenever persisting summary memory config.","Remember this buffer also needs an llm for summarization."],"tags":["llama-index","memory","summarization","validation","configuration"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}