mudler/LocalAI · error · RuntimeError

Model not loaded

Error message

Model not loaded

What it means

RuntimeError from liquid-audio backend's AudioToAudioStream handler: _audio_to_audio_stream() requires self.model and self.processor, which are only populated by a successful LoadModel gRPC call. Streaming audio-to-audio without a prior model load fails immediately; the outer handler catches it and yields an error event with the message in meta JSON.

Source

Thrown at backend/python/liquid-audio/backend.py:383

        See `backend.proto` AudioToAudioStream for the wire protocol. Audio
        is decoded once per turn here; chunked detokenization for sub-second
        TTFB is left to a future iteration once the LFM2AudioDetokenizer
        gains a streaming entry point.
        """
        try:
            yield from self._audio_to_audio_stream(request_iterator, context)
        except Exception as exc:
            print(f"AudioToAudioStream failed: {exc}", file=sys.stderr)
            print(traceback.format_exc(), file=sys.stderr)
            yield backend_pb2.AudioToAudioResponse(
                event="error",
                meta=json.dumps({"message": str(exc)}).encode("utf-8"),
            )

    def _audio_to_audio_stream(self, request_iterator, context):
        if self.model is None or self.processor is None:
            raise RuntimeError("Model not loaded")

        import torch
        import torchaudio
        from liquid_audio import ChatState

        cfg = None
        chat = None
        input_sample_rate = 16000
        output_sample_rate = 24000
        sequence = 0

        def _new_event(event, **kwargs):
            nonlocal sequence
            sequence += 1
            kwargs.setdefault("sequence", sequence)
            return backend_pb2.AudioToAudioResponse(event=event, **kwargs)

        def _ensure_chat():

View on GitHub (pinned to 44413a9d06)

Solutions

1. Send a LoadModel request (with model id) and wait for its success response before calling AudioToAudioStream
2. Check the LoadModel response for errors — if it failed, fix the load error (model id, memory, dependencies) first
3. If this happens mid-session, the model may have been unloaded; reload it before streaming again

Example fix

# before
responses = stub.AudioToAudioStream(iter(chunks))  # no LoadModel yet

# after
stub.LoadModel(backend_pb2.ModelOptions(model="LiquidAI/LFM2.5-Audio-1.5B"))
responses = stub.AudioToAudioStream(iter(chunks))

# Before streaming, confirm the model is resident via the Health/loaded-model RPC,
# or track load state client-side:
loaded = False
resp = stub.LoadModel(backend_pb2.ModelOptions(model=MODEL_ID))
loaded = resp.success  # or not resp.error, per your proto
if not loaded:
    raise RuntimeError("LoadModel failed; not starting AudioToAudioStream")

try:
    for event in stub.AudioToAudioStream(chunk_iter()):
        ...
except grpc.RpcError as e:
    if "Model not loaded" in str(e.details()):
        reload_model_and_retry()  # LoadModel then retry once
    else:
        raise

When it happens

Trigger: Calling the AudioToAudioStream gRPC method on a fresh backend process before LoadModel; calling after a LoadModel that failed midway leaving model None; calling after the model was unloaded/released.

Common situations: Client code starts streaming immediately after backend process start, assuming the model auto-loads from a startup flag; a failed load (OOM, wrong model id) leaves the backend half-initialized and the next stream call hits this.

Related errors

• dataset_source is required (path to a preprocessed dataset)
• request was cancelled
• onnx_direct engine requires both detector_onnx and recognize
• start_image is not a readable staged file
• num_frames must not be negative

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/405a17f18191eaab. Report an issue: GitHub.