unslothai/unsloth · error · RuntimeError

Timeout waiting for '{expected_type}' response (no activity

Error message

Timeout waiting for '{expected_type}' response (no activity for {timeout}s)

What it means

Raised when the response-pump loop in _wait_response exhausts its deadline without ever receiving the expected response type. Note that 'status' heartbeats reset the deadline, so this fires only after a full timeout window with no activity at all — the subprocess is alive but silent (or its responses are not matching expected_type).

Source

Thrown at studio/backend/core/inference/orchestrator.py:614

            if rtype == "status":
                logger.info("Subprocess status: %s", resp.get("message", ""))
                # Reset deadline — subprocess is still alive and working
                deadline = time.monotonic() + timeout
                continue

            if rtype == "stall":
                msg = resp.get("message", "Download stalled")
                logger.warning("Subprocess reported stall: %s", msg)
                raise DownloadStallError(msg)

            # Other response types during wait — skip
            logger.debug(
                "Skipping response type '%s' while waiting for '%s'",
                rtype,
                expected_type,
            )

        raise RuntimeError(
            f"Timeout waiting for '{expected_type}' response (no activity for {timeout}s)"
        )

    def _drain_queue(self) -> list:
        """Drain all pending responses."""
        events = []
        if self._resp_queue is None:
            return events
        while True:
            try:
                events.append(self._resp_queue.get_nowait())
            except queue.Empty:
                return events
            except (EOFError, OSError, ValueError):
                return events

    def _direct_reader(self, request_id: str):
        """Response reader for a _gen_lock generation, safe once compare exists.

View on GitHub (pinned to 203007d190)

Solutions

1. Increase the timeout argument for the load/generation call to cover cold-cache downloads.
2. Check subprocess logs to see whether it was still legitimately working (progress) or hung.
3. Verify network throughput to the model hub — slow downloads are the usual cause.
4. If the worker is genuinely wedged, restart the backend to respawn the subprocess.
5. Confirm no other thread drained the expected response from the shared queue first.

Example fix

// before
orchestrator.load_model(name, timeout=300)
// after
orchestrator.load_model(name, timeout=3600)  # cover cold-cache download

import shutil
# estimate before loading: check free disk vs repo size before a long download
assert shutil.disk_usage("/").free > 50 * 2**30, "not enough disk for model"

try:
    orchestrator.load_model(name, timeout=3600)
except RuntimeError as e:
    if "Timeout waiting for" in str(e):
        retry_with_longer_timeout(e)

When it happens

Trigger: Waiting for 'loaded' during a very slow model download/load that exceeds the timeout, a worker wedged in a non-yielding C extension, or waiting for a response type that was already consumed by another waiter.

Common situations: Large model (~70B) load exceeding the configured timeout on slow disks, a stuck download with no stall detection triggered, GPU driver hang inside the worker, or timeout passed too small for cold-cache downloads.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Subprocess error: {error_msg}
• The current inference worker did not exit and still holds GP
• Inference subprocess is not running
• llama-server embedder failed to become healthy. Last output:
• installer timed out after {timeout_seconds}s

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/059ddaa291b9e6c9. Report an issue: GitHub.