{"record":{"id":"6bd25d17b40f9e60","repo":"janhq/jan","slug":"error-message","errorCode":null,"errorMessage":"${error.message}","messagePattern":"\\$\\{error\\.message\\}","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"extensions/llamacpp-extension/src/index.ts","lineNumber":3697,"sourceCode":"\n        buffer += decoder.decode(value, { stream: true })\n\n        // Process complete lines in the buffer\n        const lines = buffer.split('\\n')\n        buffer = lines.pop() || '' // Keep the last incomplete line in the buffer\n\n        for (const line of lines) {\n          const trimmedLine = line.trim()\n          if (!trimmedLine || trimmedLine === 'data: [DONE]') {\n            continue\n          }\n\n          if (trimmedLine.startsWith('data: ')) {\n            jsonStr = trimmedLine.slice(6)\n          } else if (trimmedLine.startsWith('error: ')) {\n            jsonStr = trimmedLine.slice(7)\n            const error = JSON.parse(jsonStr)\n            throw new Error(error.message)\n          } else {\n            // it should not normally reach here\n            throw new Error('Malformed chunk')\n          }\n          try {\n            const data = JSON.parse(jsonStr)\n            const chunk = data as chatCompletionChunk\n\n            yield chunk\n          } catch (e) {\n            logger.error('Error parsing JSON from stream or server error:', e)\n            // re‑throw so the async iterator terminates with an error\n            throw e\n          }\n        }\n      }\n    } finally {\n      reader.releaseLock()","sourceCodeStart":3679,"sourceCodeEnd":3715,"githubUrl":"https://github.com/janhq/jan/blob/fad3f12a147d138388a66f0d92a02b2675f65294/extensions/llamacpp-extension/src/index.ts#L3679-L3715","documentation":"Thrown inside the SSE line-parsing loop when a line begins with 'error: '. The extension slices off the prefix, JSON.parses the remainder, and throws a new Error carrying the server-supplied error.message. This is the channel by which the llama.cpp router (or any OpenAI-compatible upstream) reports mid-stream inference errors that arrive as SSE error events rather than as HTTP status codes.","triggerScenarios":"Model fails partway through generation (OOM, NaN loss, kernel panic) and the server emits an SSE error event instead of closing the stream. Unsupported sampling combination detected mid-stream. The router forwards an upstream error from a piped model. The 'error: ' payload is malformed JSON (then JSON.parse itself throws SyntaxError, which propagates as the message).","commonSituations":"Long generation that OOMs after the headers were already sent. Quantization-specific instability producing NaN. Mid-stream abort by the router due to an internal watchdog. A proxy rewriting data: lines as error: lines.","solutions":["Inspect the embedded error.message - it is the server's own description (OOM, NaN, watchdog, etc.).","For OOM: lower batch_size/ubatch_size, context length, or use a smaller quant; unload other models.","If NaN/instability: switch sampling params (temperature, repeat_penalty) or the quant level.","Retry once - transient mid-stream errors sometimes succeed on a fresh session.","Capture router logs at the timestamp of the error for the underlying stack trace."],"exampleFix":"// before\nfor await (const c of await provider.chat(opts, ac)) { /* ... */ } // throws mid-stream\n// after - classify and recover\ntry { for await (const c of await provider.chat(opts, ac)) emit(c) }\ncatch (e) {\n  const m = String(e)\n  if (/out of memory|OOM/i.test(m)) { opts.n_gpu_layers = Math.max(0, (opts.n_gpu_layers ?? 0) - 5); /* retry */ }\n  else if (m === 'Unexpected end of JSON input') { /* 'error: ' line was not JSON - report protocol error */ throw new Error('malformed SSE error event') }\n  else throw e\n}","handlingStrategy":"try-catch","validationCode":"// Pre-validate sampling params to reduce mid-stream errors\nfunction validateSampling(opts: any) {\n  if (opts.temperature != null && (opts.temperature < 0 || opts.temperature > 2)) throw new Error('temperature out of range')\n  if (opts.top_p != null && (opts.top_p <= 0 || opts.top_p > 1)) throw new Error('top_p out of range')\n}\nvalidateSampling(opts)","typeGuard":"// Cannot type-guard a server-emitted event; validate the JSON payload shape at parse site.\nfunction isErrorEventPayload(x: unknown): x is { message: string } {\n  return typeof (x as any)?.message === 'string'\n}","tryCatchPattern":"try { for await (const c of await provider.chat(opts, ac)) emit(c) }\ncatch (e) {\n  const m = String(e)\n  if (/out of memory|OOM/i.test(m)) { opts.n_gpu_layers = Math.max(0, (opts.n_gpu_layers ?? 0) - 5); /* retry */ }\n  else if (/Unexpected end of JSON input/.test(m)) throw new Error('malformed SSE error event from server')\n  else throw e\n}","preventionTips":["Conservatively bound context/batch sizes to avoid mid-generation OOM.","Use stable sampling params; avoid exotic combinations your llama.cpp version doesn't support.","Capture router logs at inference time so mid-stream errors have a matching server-side trace."],"tags":["chat","streaming","sse","inference","runtime-error","llama-server"],"backgroundTag":null,"analyzedSha":"fad3f12a147d138388a66f0d92a02b2675f65294","analyzedAt":"2026-08-12T20:33:47.516Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}