zylon-ai/private-gpt · error · ValueError

Token limit must be set and greater than 0.

Error message

Token limit must be set and greater than 0.

What it means

Raised by TrimmingMemory's pydantic model_validator(mode='before') when token_limit is missing or below 1 — values.get('token_limit', -1) means an absent token_limit reads as -1 and fails immediately. This is deliberate fail-fast validation: a trimming memory without a positive budget cannot trim, so construction is rejected instead of silently misbehaving at runtime.

Source

Thrown at private_gpt/components/memory/trimming_memory.py:80

        exclude=True,
    )
    tokenizer_fn: TokenizerFn = Field(
        exclude=True,
    )

    @classmethod
    def class_name(cls) -> str:
        """Get class name."""
        return "TrimmingMemory"

    @model_validator(mode="before")
    @classmethod
    def validate_memory(cls, values: dict[str, Any]) -> dict[str, Any]:
        """Validate memory configuration."""
        # Validate token limit
        token_limit = values.get("token_limit", -1)
        if token_limit < 1:
            raise ValueError("Token limit must be set and greater than 0.")

        # Validate tokenizer
        tokenizer_fn = values.get("tokenizer_fn")
        if tokenizer_fn is None:
            # TODO: Replace with a default tokenizer function
            raise ValueError("tokenizer_fn must be provided.")

        # Validate text splitter
        text_splitter = values.get("text_splitter")
        if text_splitter is None:
            values["text_splitter"] = _default_text_splitter

        # Validate strategy-specific constraints
        trim_strategy = values.get("trim_strategy", TrimStrategy.LAST)
        start_on = values.get("start_on")
        include_system = values.get("include_system", True)

        if start_on and trim_strategy == TrimStrategy.FIRST:

View on GitHub (pinned to 4a030776a3)

Solutions

1. Pass a positive token_limit (e.g. 2048) when constructing TrimmingMemory.
2. Prefer from_defaults, which derives token_limit from llm.metadata.context_window * DEFAULT_TOKEN_LIMIT_RATIO or DEFAULT_TOKEN_LIMIT when not given.
3. Fix the upstream value if context_window is 0 (see error 156's sibling path) — check the LLM metadata source.
4. Validate token_limit > 0 in your config loader before constructing memory.

Example fix

# before
memory = TrimmingMemory(token_limit=0, tokenizer_fn=tok)

# after
memory = TrimmingMemory(token_limit=int(llm.metadata.context_window * 0.75), tokenizer_fn=tok)

token_limit = token_limit or int(llm.metadata.context_window * 0.75)
assert token_limit >= 1, 'token_limit must be positive'

try:
    mem = TrimmingMemory(token_limit=tl, tokenizer_fn=tok)
except ValidationError as e:
    if 'Token limit' in str(e):
        tl = DEFAULT_TOKEN_LIMIT; mem = TrimmingMemory(token_limit=tl, tokenizer_fn=tok)

When it happens

Trigger: Constructing TrimmingMemory(token_limit=0), TrimmingMemory(token_limit=-100), or omitting token_limit entirely when instantiating the model directly (bypassing from_defaults, which derives a limit from the LLM context window or DEFAULT_TOKEN_LIMIT).

Common situations: Direct model instantiation in tests or custom wiring without a limit; config where token_limit is parsed as 0 (e.g. unset env var coerced to int); passing context_window-derived limits from an LLM that reports 0.

Related errors

• Unknown memory type: {type}
• tokenizer_fn must be provided.
• start_on can only be used with 'last' strategy
• include_system can only be used with 'last' strategy
• must be a list or comma-separated string

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/5e1ba5b8b56f1927. Report an issue: GitHub.