janhq/jan · error · Error

API request failed with status ${response.status}: ${JSON.st

Error message

API request failed with status ${response.status}: ${JSON.stringify(errorData)}

What it means

Thrown by handleStreamingResponse() when the streaming POST to the router's /v1/chat/completions returns a non-2xx status. The response body is parsed as JSON (errorData) and embedded verbatim into the message. This wraps any HTTP-layer failure from the llama.cpp server (model error, bad request, internal panic, timeout-abort) into a single descriptive error for the streaming path.

Source

Thrown at extensions/llamacpp-extension/src/index.ts:3657

        combinedController.abort(abortController.signal.reason)
      } else {
        abortController.signal.addEventListener(
          'abort',
          () => combinedController.abort(abortController.signal.reason),
          { once: true }
        )
      }
    }
    const response = await fetch(url, {
      method: 'POST',
      headers,
      body,
      connectTimeout: Number(this.timeout) * 1000, // default 10 minutes
      signal: combinedController.signal,
    }).finally(() => clearTimeout(timeoutId))
    if (!response.ok) {
      const errorData = await response.json().catch(() => null)
      throw new Error(
        `API request failed with status ${response.status}: ${JSON.stringify(
          errorData
        )}`
      )
    }

    if (!response.body) {
      throw new Error('Response body is null')
    }

    const reader = response.body.getReader()
    const decoder = new TextDecoder('utf-8')
    let buffer = ''
    let jsonStr = ''
    try {
      while (true) {
        const { done, value } = await reader.read()

View on GitHub (pinned to fad3f12a14)

Solutions

1. Read errorData in the message - it usually contains the llama-server error string telling you the exact cause (OOM, bad param, etc.).
2. If OOM/context-too-long: reduce context length, unload other models, or use a smaller quant.
3. If 400 bad request: inspect the request body for unsupported keys (template_kwargs, sampling) against your llama.cpp version.
4. If timeout/abort: raise this.timeout or avoid aborting unless intentional.
5. If 503/server panic: capture router logs, restart the router, and report the panic.

Example fix

// before
const stream = await provider.chat(opts, ac) // throws on HTTP 500
// after - surface server's error and retry on context error
try { for await (const c of await provider.chat(opts, ac)) yield c }
catch (e) {
  const m = String(e)
  if (/context.+length|too long/i.test(m)) { opts.ctx_size = Math.min((opts.ctx_size ?? 4096) * 2, 32768); /* retry */ }
  else throw e
}

// Pre-validate request shape before sending (catch common 400s)
function validateChatOpts(opts: any) {
  if (typeof opts.model !== 'string' || !opts.model) throw new Error('opts.model required')
  if (opts.messages && !Array.isArray(opts.messages)) throw new Error('opts.messages must be an array')
  if (opts.max_tokens != null && (!Number.isFinite(opts.max_tokens) || opts.max_tokens <= 0)) throw new Error('bad max_tokens')
}
validateChatOpts(opts)

function isChatCompletionRequest(x: unknown): x is { model: string; messages: unknown[] } {
  return typeof (x as any)?.model === 'string' && Array.isArray((x as any)?.messages)
}

try { for await (const c of await provider.chat(opts, ac)) emit(c) }
catch (e) {
  const m = String(e)
  if (/status 4\d\d/.test(m)) { /* fix request from errorData, do not retry blindly */ throw new Error('bad request: ' + m) }
  if (/status 5\d\d|timed out/i.test(m)) { /* transient - one retry */ }
  else throw e
}

When it happens

Trigger: Model OOM / context too long (HTTP 500 from llama-server). Malformed request body (400). The router aborted (503). The combined AbortController fired (timeout or user cancel) producing a non-ok response. Authentication failure if api_key was wrong. Server panicked on an unsupported sampling/template parameter.

Common situations: Context window exceeded for the loaded model. Prompt template/chat_template kwargs sent by the client are incompatible with the model. Sampling params (e.g. min_p with an old backend) unsupported. Timeout too short for a long generation. The combined signal aborted mid-flight due to the user stopping generation.

Related errors

• ${error.message}
• Response body is null
• Malformed chunk
• MLX API request failed with status ${response.status}: ${JSO
• Response body is null

AI-assisted analysis of janhq/jan@fad3f12a14 (2026-08-12). Data as JSON: /api/errors/c950d57c1b565a83. Report an issue: GitHub.