unslothai/unsloth · error · RuntimeError

No active model

Error message

No active model

What it means

The inner chat-generation routine raises RuntimeError('No active model') when self.active_model_name is falsy — i.e. no model has been loaded/activated on the engine before generation is attempted. Every generation path dereferences self.models[self.active_model_name], so the guard prevents a KeyError and gives a clear message.

Source

Thrown at studio/backend/core/inference/inference.py:1112

        max_new_tokens: int = 256,
        repetition_penalty: float = 1.0,
        cancel_event = None,
        _adapter_state = None,
        tools: Optional[list] = None,
        enable_thinking: Optional[bool] = None,
        reasoning_effort: Optional[str] = None,
        preserve_thinking: Optional[bool] = None,
        continue_final_message: bool = False,
        presence_penalty: float = 0.0,
    ) -> Generator[str, None, None]:
        """Inner generation logic, called by generate_chat_response and
        generate_with_adapter_control.

        _adapter_state is passed to generate_stream/vision so the background
        thread can toggle adapters under the generation lock.
        """
        if not self.active_model_name:
            raise RuntimeError("No active model")

        model_info = self.models[self.active_model_name]
        is_vision = model_info.get("is_vision", False)
        tokenizer = model_info.get("tokenizer") or model_info.get("processor")
        # Unwrap processor -> raw tokenizer for VLMs on the text path.
        tokenizer = getattr(tokenizer, "tokenizer", tokenizer)
        top_k = self._normalize_top_k(top_k)

        if is_vision and image:
            # Verify the stored processor can handle images; FastVisionModel may
            # return a raw tokenizer instead of a ProcessorMixin (e.g. Gemma-3).
            from transformers import ProcessorMixin

            processor = model_info.get("processor")
            has_image_processing = processor is not None and (
                isinstance(processor, ProcessorMixin) or hasattr(processor, "image_processor")
            )
            if has_image_processing:

View on GitHub (pinned to 203007d190)

Solutions

1. Load and activate a model first (await the load endpoint) before sending generation requests
2. If a load failed, address that failure (see load error) and retry the load
3. On the client, gate the send button on the engine's 'model loaded' state

Example fix

// before
for tok in engine._generate(...): ...  # no model loaded
// after
engine.load_model("qwen2.5-7b-instruct")
for tok in engine._generate(...): ...

if not getattr(engine, "active_model_name", None):
    raise HTTPException(409, "No model is loaded — load one before generating")

def has_active_model(engine) -> bool:
    return bool(getattr(engine, "active_model_name", None)) and engine.active_model_name in engine.models

try:
    yield from engine._generate(...)
except RuntimeError as e:
    if str(e) == "No active model":
        return JSONResponse(status_code=409, content={"detail": "Load a model first"})
    raise

When it happens

Trigger: Calling generate_chat_response / generate_stream before any load_model succeeded, after unload_model cleared the active model, or after a failed load left active_model_name unset.

Common situations: Service restart losing loaded state while a queued request arrives; frontend allows sending chat before load completes; a previous load failed and the client ignored the error and sent a generation anyway.

Related errors

• deadline reached while pacing before {method} {_redact_url(u
• deadline reached before {method} {_redact_url(url)}
• VirusTotal returned HTTP {status} for {_redact_url(url)}
• VirusTotal request failed after {max_attempts} attempt(s): {
• VirusTotal hash lookup returned a malformed body

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/9e0d4d8eb99f3ec4. Report an issue: GitHub.