janhq/jan · critical · Error

llama.cpp router is not running. Please restart the app.

Error message

llama.cpp router is not running. Please restart the app.

What it means

Thrown by performLoad() after ensureRouterReady() (which itself tries ensureProvisioned + getRouterInfo, then startRouter if down) and a second getRouterInfo() still returns null. The llama.cpp router is the sidecar process that owns model sessions; without it, loadLlamaModel cannot be issued. This error means the router failed to start or crashed during startup and could not be recovered.

Source

Thrown at extensions/llamacpp-extension/src/index.ts:3468

  // still isn't up. Safe to call redundantly: startRouter reuses a router that
  // already matches this config rather than respawning it.
  private async ensureRouterReady(): Promise<void> {
    // Provisions if the setup screen never got the chance to ask, so skipping
    // setup and later loading a local model still works.
    await this.ensureProvisioned().catch(() => undefined)
    if (!(await this.getRouterInfo())) {
      await this.startRouter()
    }
  }

  private async performLoad(
    modelId: string,
    isEmbedding: boolean = false
  ): Promise<SessionInfo> {
    await this.ensureRouterReady()
    const router = await this.getRouterInfo()
    if (!router) {
      throw new Error(
        'llama.cpp router is not running. Please restart the app.'
      )
    }

    if (!isEmbedding) {
      await this.evictChatIfAtCapacity(modelId)
    }

    try {
      const info = await loadLlamaModel(modelId, isEmbedding)
      if (!isEmbedding) {
        this.loadedChatOrder = this.loadedChatOrder.filter((m) => m !== modelId)
        this.loadedChatOrder.push(modelId)
      }
      return info
    } catch (error) {
      logger.error('Error in load command:\n', error)
      throw error

View on GitHub (pinned to fad3f12a14)

Solutions

1. Restart the app as the message suggests - a transient router crash on startup often recovers on relaunch.
2. Re-provision the backend (installBackend) to ensure the matching llama-server binary exists and is executable.
3. Check the logs for the router's stderr - look for missing libs, port-in-use, or permission errors and address the specific cause.
4. Free up the port range or kill orphaned llama-server processes left by a previous crash.
5. On Linux, install missing dynamic loader deps (ldd on the binary); on Windows, install the VC++ redistributable / CUDA runtime.

Example fix

// before
await provider.load('qwen') // throws: router not running
// after
let info = await provider.getRouterInfo()
if (!info) {
  await provider.ensureProvisioned()
  await provider.startRouter()
  info = await provider.getRouterInfo()
}
if (!info) throw new Error('router still down - check logs for root cause')
await provider.load('qwen')

// Best-effort router liveness check with explicit retry before failing
async function ensureRouterUp(provider: any, retries = 3) {
  for (let i = 0; i < retries; i++) {
    if (await provider.getRouterInfo()) return
    await provider.ensureProvisioned().catch(() => {})
    await provider.startRouter()
  }
  throw new Error('router could not be started after retries - inspect logs')
}

async function routerIsAlive(provider: { getRouterInfo(): Promise<unknown> }): Promise<boolean> {
  try { return Boolean(await provider.getRouterInfo()) } catch { return false }
}

try { await provider.load(modelId) }
catch (e) {
  if (/router is not running/.test(String(e))) { await restartAppOrRouter(); await provider.load(modelId) }
  else throw e
}

When it happens

Trigger: Router binary is missing, not executable, or the wrong arch (e.g. arm binary on x86). Router crashed on startup due to a port conflict (getRandomPort range collision), missing system libs, or a malformed config. Provisioning/install never completed so the binary isn't on disk. A security tool blocked the router process. Antivirus quarantined llama-server.

Common situations: First run after install where backend provisioning silently failed. OS update removed a runtime dependency (libc, Vulkan loader). Another app holds the ephemeral port the router picked. Corporate endpoint protection blocks the unsigned sidecar. User moved the data folder and the relative path to the router binary no longer resolves.

Related errors

• Router failed its health check on backend ${targetBackendStr
• Model already loaded!!
• ${String(e)}
• RAG extension not available

AI-assisted analysis of janhq/jan@fad3f12a14 (2026-08-12). Data as JSON: /api/errors/b1a7fe77623da9fe. Report an issue: GitHub.