oobabooga/textgen · error · ValueError

No tokenizer is loaded

Error message

No tokenizer is loaded

What it means

Error "No tokenizer is loaded" thrown in oobabooga/textgen.

Source

Thrown at modules/text_generation.py:135

                if (cur_time - last_update) > latency_threshold:
                    yield reply
                last_update = time.monotonic()

        stop_event = state.get('stop_event')
        if stop_found or shared.stop_everything or (stop_event and stop_event.is_set()):
            break

    if not is_chat:
        reply = apply_extensions('output', reply, state)

    yield reply


def encode(prompt, add_special_tokens=True, add_bos_token=True, truncation_length=None):
    if shared.tokenizer is None:
        models.load_model_if_idle_unloaded()
        if shared.tokenizer is None:
            raise ValueError('No tokenizer is loaded')

    # llama.cpp case
    if shared.model.__class__.__name__ == 'LlamaServer':
        input_ids = shared.tokenizer.encode(str(prompt), add_bos_token=add_bos_token)
        input_ids = np.array(input_ids).reshape(1, len(input_ids))

        if truncation_length is not None:
            input_ids = input_ids[:, -truncation_length:]

        return input_ids

    # All other model types
    else:
        import torch

        from modules.torch_utils import get_device

        if shared.model.__class__.__name__ in ['Exllamav3Model', 'TensorRTLLMModel']:

View on GitHub (pinned to ed888c71f2)

Solutions

1. Load a model (with its tokenizer) via the UI or API before requesting tokenization or generation.

When it happens

Trigger: Raised when text generation or tokenization is requested but no model/tokenizer is currently loaded in the web UI. Triggers when calling generate before loading a model, or after the model was unloaded.

Common situations: See trigger scenarios.

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/bbc602222584537c. Report an issue: GitHub.