zylon-ai/private-gpt · error · ValueError

Empty response from Mistral tokenizer

Error message

Empty response from Mistral tokenizer

What it means

In apply_chat_template with tokenize=False, the wrapper returns encoded.text from mistral_common's encode_chat_completion. If the encoding produced None or an empty string — which should not happen for a valid request — it raises ValueError('Empty response from Mistral tokenizer') to avoid returning a falsy prompt.

Source

Thrown at private_gpt/components/llm/tokenizers/mistral.py:662

        request: Any = ChatCompletionRequest(
            messages=messages,  # type: ignore[arg-type]
            tools=[Tool(**tool) for tool in tools] if tools else None,
        )

        # Apply pydantic workaround
        maybe_serialize_tool_calls(request)
        truncate_tool_call_ids(request)

        encoded = self.mistral.encode_chat_completion(request)

        if tokenize:
            tokens: list[int] = encoded.tokens
            return tokens
        else:
            result = cast(str | None, encoded.text)
            if not result:
                raise ValueError("Empty response from Mistral tokenizer")
            return result

    def decode(
        self,
        ids: list[int] | int,
        skip_special_tokens: bool = True,
    ) -> str:
        """Decode token IDs to text."""
        if isinstance(ids, int):
            ids = [ids]

        if not skip_special_tokens:
            SpecialTokenPolicy = _load_mistral_module(
                "mistral_common.tokens.tokenizers.base"
            ).SpecialTokenPolicy

            return cast(str, self.tokenizer.decode(ids, SpecialTokenPolicy.KEEP))

View on GitHub (pinned to 4a030776a3)

Solutions

1. Inspect the request passed to encode_chat_completion: verify messages are non-empty and have non-empty content after normalization.
2. Pin mistral-common to the version validated by the project (reinstall via uv sync --inexact --extra llm-mistral).
3. Reproduce with tokenize=True to see whether the encoding itself works, narrowing the issue to text rendering.

Example fix

# before
prompt = tok.apply_chat_template([], tokenize=False)  # degenerate input

# after
assert messages and all(m.get('content') for m in messages if m['role'] != 'tool')
prompt = tok.apply_chat_template(messages, tokenize=False)

def prompt_input_ok(messages) -> bool:
    return bool(messages) and any(
        m.get('role') in ('user', 'system', 'assistant', 'tool') and (m.get('content') or m.get('tool_calls'))
        for m in messages
    )

try:
    prompt = tok.apply_chat_template(messages, tokenize=False)
except ValueError as e:
    if 'Empty response' in str(e):
        logger.error('degenerate encode for messages=%r', messages)
        raise BadPromptError('conversation produced empty template output') from e
    raise

When it happens

Trigger: Calling the mistral tokenizer's apply_chat_template(..., tokenize=False) on a request whose encoded form has no text: malformed/degenerate message lists (e.g. empty messages, content stripped to nothing after normalization) or version drift in mistral_common changing encode output.

Common situations: Edge-case inputs such as empty conversation arrays; messages whose content is removed by normalization (reasoning stripped, tool-call content empty); mismatches between the wrapper's request building and the installed mistral-common version.

Related errors

• Mistral tokenizer dependencies are not installed. Install wi
• Found {len(matched_files)} files matching the pattern: {file
• Found {len(matched_files)} files matching the pattern: {file
• Mistral tokenizer must be in test mode. Set `mode=Validation
• Unsupported tokenizer: {type(self.tokenizer)}

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/45b391bc82e1efc4. Report an issue: GitHub.