HKUDS/DeepTutor · error · LLMAPIError

Local LLM failed: streaming={e}, non-streaming={e2}

Error message

Local LLM failed: streaming={e}, non-streaming={e2}

What it means

stream() falls back to non-streaming complete() when streaming fails; if both attempts raise, the provider gives up and wraps the combined failure in LLMAPIError (provider='local') without leaking either traceback.

Source

Thrown at deeptutor/services/llm/local_provider.py:373

        raise  # Re-raise LLM errors as-is
    except Exception as e:
        # Streaming failed, fall back to non-streaming
        logger.warning("Streaming failed (%s), falling back to non-streaming", e)

        try:
            content = await complete(
                prompt=prompt,
                system_prompt=system_prompt,
                model=model,
                api_key=api_key,
                base_url=base_url,
                messages=messages,
                **kwargs,
            )
            if content:
                yield content
        except Exception as e2:
            raise LLMAPIError(
                f"Local LLM failed: streaming={e}, non-streaming={e2}",
                provider="local",
            )


async def fetch_models(
    base_url: str,
    api_key: str | None = None,
) -> list[str]:
    """
    Fetch available models from local LLM server.

    Supports:
    - Ollama (/api/tags)
    - OpenAI-compatible (/models)

    Args:
        base_url: Base URL for the local server

View on GitHub (pinned to 3e82f13042)

Solutions

1. Check that the local LLM server is running and reachable (curl the base_url).
2. Fix base_url/model configuration if both paths fail immediately.
3. Check server logs for crashes (OOM is common with large models).

import socket
from urllib.parse import urlparse

def endpoint_reachable(base_url: str, timeout=2) -> bool:
    p = urlparse(base_url)
    try:
        socket.create_connection((p.hostname, p.port or 80), timeout=timeout).close()
        return True
    except OSError:
        return False

try:
    async for chunk in provider.stream(messages): ...
except LLMAPIError:
    switch_to_backup_provider_or_report('local server unreachable')

When it happens

Trigger: Streaming raises (network drop, non-200) and the subsequent non-streaming retry also raises — e.g. the local server went down mid-request, or the endpoint is entirely unreachable so both paths fail identically.

Common situations: Local server crashed or was restarted during a session; wrong port/URL so both connection attempts fail; firewall/DNS failure.

Related errors

• Channel onboarding provider request failed ({type(exc).__nam
• OpenAI SDK connection error: {exc}
• Image generation request error: {exc}
• Image generation request error: {exc}
• Connection to {binding} API was forcibly closed. This may in

AI-assisted analysis of HKUDS/DeepTutor@3e82f13042 (2026-08-27). Data as JSON: /api/errors/0011e7d6c88e008b. Report an issue: GitHub.