{"record":{"id":"cb97b956face6bac","repo":"janhq/jan","slug":"tokenize-request-failed-with-status-res-status","errorCode":null,"errorMessage":"Tokenize request failed with status ${res.status}","messagePattern":"Tokenize request failed with status (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"extensions/llamacpp-extension/src/index.ts","lineNumber":4174,"sourceCode":"   * on its session port. Char-based chunking can't reliably predict token\n   * count (subword tokenizers vary widely by content), so callers that need\n   * a hard guarantee against exceed_context_size_error should verify with\n   * this rather than estimating from character length.\n   */\n  async countEmbeddingTokens(texts: string[]): Promise<number[]> {\n    const sInfo = await this.ensureEmbeddingModelLoaded()\n    const counts: number[] = []\n    for (const text of texts) {\n      const res = await fetch(`http://localhost:${sInfo.port}/tokenize`, {\n        method: 'POST',\n        headers: {\n          'Content-Type': 'application/json',\n          'Authorization': `Bearer ${sInfo.api_key}`,\n        },\n        body: JSON.stringify({ content: text, model: sInfo.model_id }),\n      })\n      if (!res.ok) {\n        throw new Error(`Tokenize request failed with status ${res.status}`)\n      }\n      const json = (await res.json()) as { tokens?: unknown[] }\n      counts.push(Array.isArray(json.tokens) ? json.tokens.length : 0)\n    }\n    return counts\n  }\n\n  async embed(text: string[]): Promise<EmbeddingResponse> {\n    const sInfo = await this.ensureEmbeddingModelLoaded()\n\n    const ubatchSize =\n      (this.config?.ubatch_size && this.config.ubatch_size > 0\n        ? this.config.ubatch_size\n        : 512) || 512\n    const batches = buildEmbedBatches(text, ubatchSize)\n\n    const attemptRequest = async (\n      session: SessionInfo,","sourceCodeStart":4156,"sourceCodeEnd":4192,"githubUrl":"https://github.com/janhq/jan/blob/fad3f12a147d138388a66f0d92a02b2675f65294/extensions/llamacpp-extension/src/index.ts#L4156-L4192","documentation":"Thrown by countEmbeddingTokens() when the embedding model session's /tokenize HTTP endpoint returns a non-OK status. The session (sInfo) was resolved by ensureEmbeddingModelLoaded(), so a model is nominally loaded; the failure is that the in-process llamacpp server is not responding correctly to this specific request.","triggerScenarios":"The embedding model process is starting up and not yet ready; the session crashed but findSessionByModel still returned stale info; an empty/oversized 'content' body the tokenizer rejects; the api_key header is wrong and returns 401/403.","commonSituations":"Calling countEmbeddingTokens immediately after load before /tokenize is live; concurrent unload invalidated the port; model_id in the body does not match the loaded session; port collision where another process now owns sInfo.port.","solutions":["Retry after a short delay or poll http://localhost:<port>/health until 200 before tokenizing.","Confirm the embedding model session is still loaded (findSessionByModel) and reload if it was unloaded.","Check that sInfo.api_key and sInfo.model_id match the running session.","Inspect the response body/status (401 vs 500) — 401 means auth, 5xx means the server process is unhealthy."],"exampleFix":"// before\nconst res = await fetch(url, { method: 'POST', headers, body })\nif (!res.ok) throw new Error(`Tokenize request failed with status ${res.status}`)\n\n// after\nconst res = await fetch(url, { method: 'POST', headers, body })\nif (!res.ok) {\n  const detail = await res.text().catch(() => '')\n  throw new Error(`Tokenize failed (${res.status}) on port ${sInfo.port}: ${detail}`)\n}","handlingStrategy":"retry","validationCode":"async function tokenizeEndpointReady(port: number): Promise<boolean> {\n  try {\n    const r = await fetch(`http://localhost:${port}/health`)\n    return r.ok\n  } catch { return false }\n}\n\nif (!(await tokenizeEndpointReady(sInfo.port))) {\n  throw new Error('Embedding /tokenize not ready; wait for health')\n}","typeGuard":"function hasSessionPort(s: unknown): s is { port: number; api_key: string; model_id: string } {\n  return typeof s === 'object' && s !== null\n    && typeof (s as any).port === 'number'\n    && typeof (s as any).api_key === 'string'\n    && typeof (s as any).model_id === 'string'\n}","tryCatchPattern":"for (let attempt = 0; attempt < 3; attempt++) {\n  const res = await fetch(url, { method: 'POST', headers, body })\n  if (res.ok) { /* proceed */ break }\n  if (res.status >= 500 && attempt < 2) { await delay(500 * (attempt + 1)); continue }\n  throw new Error(`Tokenize failed ${res.status}`)\n}","preventionTips":["Poll /health before calling countEmbeddingTokens after a fresh load.","Reload the embedding model if its session was unloaded between calls.","Verify api_key/model_id match the live session."],"tags":["http","embedding","tokenizer","session-readiness","typescript"],"backgroundTag":null,"analyzedSha":"fad3f12a147d138388a66f0d92a02b2675f65294","analyzedAt":"2026-08-12T20:33:47.516Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}