What causes "${error.message}" in janhq/jan?

Inspect the embedded error.message - it is the server's own description (OOM, NaN, watchdog, etc.).

janhq/jan · error · Error

${error.message}

Error message

${error.message}

What it means

Thrown inside the SSE line-parsing loop when a line begins with 'error: '. The extension slices off the prefix, JSON.parses the remainder, and throws a new Error carrying the server-supplied error.message. This is the channel by which the llama.cpp router (or any OpenAI-compatible upstream) reports mid-stream inference errors that arrive as SSE error events rather than as HTTP status codes.

Source

Thrown at extensions/llamacpp-extension/src/index.ts:3697

        buffer += decoder.decode(value, { stream: true })

        // Process complete lines in the buffer
        const lines = buffer.split('\n')
        buffer = lines.pop() || '' // Keep the last incomplete line in the buffer

        for (const line of lines) {
          const trimmedLine = line.trim()
          if (!trimmedLine || trimmedLine === 'data: [DONE]') {
            continue
          }

          if (trimmedLine.startsWith('data: ')) {
            jsonStr = trimmedLine.slice(6)
          } else if (trimmedLine.startsWith('error: ')) {
            jsonStr = trimmedLine.slice(7)
            const error = JSON.parse(jsonStr)
            throw new Error(error.message)
          } else {
            // it should not normally reach here
            throw new Error('Malformed chunk')
          }
          try {
            const data = JSON.parse(jsonStr)
            const chunk = data as chatCompletionChunk

            yield chunk
          } catch (e) {
            logger.error('Error parsing JSON from stream or server error:', e)
            // re‑throw so the async iterator terminates with an error
            throw e
          }
        }
      }
    } finally {
      reader.releaseLock()

View on GitHub (pinned to fad3f12a14)

Solutions

Inspect the embedded error.message - it is the server's own description (OOM, NaN, watchdog, etc.).
For OOM: lower batch_size/ubatch_size, context length, or use a smaller quant; unload other models.
If NaN/instability: switch sampling params (temperature, repeat_penalty) or the quant level.
Retry once - transient mid-stream errors sometimes succeed on a fresh session.
Capture router logs at the timestamp of the error for the underlying stack trace.

Example fix

// before
for await (const c of await provider.chat(opts, ac)) { /* ... */ } // throws mid-stream
// after - classify and recover
try { for await (const c of await provider.chat(opts, ac)) emit(c) }
catch (e) {
  const m = String(e)
  if (/out of memory|OOM/i.test(m)) { opts.n_gpu_layers = Math.max(0, (opts.n_gpu_layers ?? 0) - 5); /* retry */ }
  else if (m === 'Unexpected end of JSON input') { /* 'error: ' line was not JSON - report protocol error */ throw new Error('malformed SSE error event') }
  else throw e
}

Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-validate sampling params to reduce mid-stream errors
function validateSampling(opts: any) {
  if (opts.temperature != null && (opts.temperature < 0 || opts.temperature > 2)) throw new Error('temperature out of range')
  if (opts.top_p != null && (opts.top_p <= 0 || opts.top_p > 1)) throw new Error('top_p out of range')
}
validateSampling(opts)

Type guard

// Cannot type-guard a server-emitted event; validate the JSON payload shape at parse site.
function isErrorEventPayload(x: unknown): x is { message: string } {
  return typeof (x as any)?.message === 'string'
}

Try / catch

try { for await (const c of await provider.chat(opts, ac)) emit(c) }
catch (e) {
  const m = String(e)
  if (/out of memory|OOM/i.test(m)) { opts.n_gpu_layers = Math.max(0, (opts.n_gpu_layers ?? 0) - 5); /* retry */ }
  else if (/Unexpected end of JSON input/.test(m)) throw new Error('malformed SSE error event from server')
  else throw e
}

Prevention

Conservatively bound context/batch sizes to avoid mid-generation OOM.
Use stable sampling params; avoid exotic combinations your llama.cpp version doesn't support.
Capture router logs at inference time so mid-stream errors have a matching server-side trace.

When it happens

Trigger: Model fails partway through generation (OOM, NaN loss, kernel panic) and the server emits an SSE error event instead of closing the stream. Unsupported sampling combination detected mid-stream. The router forwards an upstream error from a piped model. The 'error: ' payload is malformed JSON (then JSON.parse itself throws SyntaxError, which propagates as the message).

Common situations: Long generation that OOMs after the headers were already sent. Quantization-specific instability producing NaN. Mid-stream abort by the router due to an internal watchdog. A proxy rewriting data: lines as error: lines.

Related errors

AI-assisted analysis of janhq/jan@fad3f12a14 (2026-08-12). Data as JSON: /api/errors/6bd25d17b40f9e60. Report an issue: GitHub.