unslothai/unsloth · error · SttEngineUnavailableError

The local transcription runtime did not answer the request.

Error message

The local transcription runtime did not answer the request. Transcription will use the Transformers engine from now on.

What it means

SttEngineUnavailableError raised when the /inference request fails at the transport level (connection refused/reset, timeout, JSON decode of the response): any non-SttAudioDecodeError/SttEngineUnavailableError exception is wrapped in this error. The raising call also records a runtime inference failure (unless a cancel caused the socket close), which permanently routes future transcriptions to the Transformers engine until a success calls clear_runtime_inference_failure().

Source

Thrown at studio/backend/core/inference/stt_ggml_sidecar.py:1202

                "POST",
                "/inference",
                body = body,
                headers = {"Content-Type": f"multipart/form-data; boundary={boundary}"},
            )
            with connection.getresponse() as response:
                if not 200 <= response.status < 300:
                    raise SttEngineUnavailableError(
                        f"The local transcription runtime returned HTTP {response.status}."
                    )
                payload = json.loads(response.read().decode("utf-8"))
        except (SttAudioDecodeError, SttEngineUnavailableError):
            raise
        except Exception as exc:
            # A cancel closes this socket deliberately, so it is not evidence of a broken
            # runtime and must not disable the engine.
            if cancel_event is None or not cancel_event.is_set():
                note_runtime_inference_failure(f"{type(exc).__name__}: {exc}")
            raise SttEngineUnavailableError(
                "The local transcription runtime did not answer the request. "
                "Transcription will use the Transformers engine from now on."
            ) from exc
        finally:
            cancel_done.set()
            connection.close()
        text = payload.get("text")
        if not isinstance(text, str):
            raise SttAudioDecodeError("Could not decode the audio.")
        # It served a transcription, so whatever failed earlier was transient.
        clear_runtime_inference_failure()
        # whisper.cpp joins segments with newlines; dictation wants one line.
        return " ".join(part.strip() for part in text.splitlines() if part.strip()).strip()


_sidecar: Optional[GgmlSttSidecar] = None

View on GitHub (pinned to 203007d190)

Solutions

Retry with the Transformers engine — the message states the fallback is already active.
Restart Studio (or trigger a model reload) to respawn whisper-server, then retry GGML dictation to clear the failure mark.
If it recurs, check RAM/VRAM headroom — a child killed mid-inference is the most common transport failure.
Keep whisper.cpp updated via `unsloth studio update`.

Defensive patterns

Strategy: fallback

Try / catch

try:
    result = sidecar.transcribe(audio, language=lang)
except SttEngineUnavailableError as exc:
    if "Transformers engine" in str(exc):
        result = transformers_stt.transcribe(audio, language=lang)  # fallback is intended

Prevention

Keep the Transformers STT engine available as the fallback path.
Monitor RAM/VRAM so whisper-server is not OOM-killed mid-request.
After restarting the engine, run one successful GGML transcription to clear the failure mark (clear_runtime_inference_failure).

When it happens

Trigger: _post_inference's connection.request/getresponse raises — whisper-server crashed mid-request, the port went away, or the response body was not parseable JSON — and cancel_event is None or not set.

Common situations: whisper-server child OOM-killed during a long transcription; server process died between load and inference; socket closed by cancel (excluded from failure-marking by design); incompatible server build returning non-JSON.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/96fa8a86c4766d05. Report an issue: GitHub.