janhq/jan · error · Error

Failed to count embedding tokens: ${e instanceof Error ? e.m

Error message

Failed to count embedding tokens: ${e instanceof Error ? e.message : String(e)}

What it means

Thrown by VectorDBExtension.splitChunkToFit() when llm.countEmbeddingTokens([text]) rejects during recursive chunk halving. Like the context-size probe, a token-count failure signals an unhealthy embedding engine; the code refuses to emit an unverified chunk that could blow past the context window. The error wraps the underlying cause.

Source

Thrown at extensions/vector-db-extension/src/index.ts:227

      return await llm.getEmbeddingContextSize!()
    } catch (e) {
      throw new Error(
        `Failed to determine embedding context size: ${e instanceof Error ? e.message : String(e)}`
      )
    }
  }

  private async splitChunkToFit(
    text: string,
    budget: number,
    llm: { countEmbeddingTokens: (texts: string[]) => Promise<number[]> }
  ): Promise<string[]> {
    if (!text) return []
    let count: number
    try {
      ;[count] = await llm.countEmbeddingTokens([text])
    } catch (e) {
      throw new Error(
        `Failed to count embedding tokens: ${e instanceof Error ? e.message : String(e)}`
      )
    }
    if (count <= budget || text.length <= MIN_CHUNK_SIZE_CHARS) return [text]
    const mid = Math.floor(text.length / 2)
    return [
      ...(await this.splitChunkToFit(text.slice(0, mid), budget, llm)),
      ...(await this.splitChunkToFit(text.slice(mid), budget, llm)),
    ]
  }

  private getEmbeddingEngine() {
    return window.core?.extensionManager.getByName('@janhq/llamacpp-extension') as AIEngine & {
      embed?: (texts: string[]) => Promise<{ data: Array<{ embedding: number[]; index: number }> }>
      getEmbeddingContextSize?: () => Promise<number | undefined>
      countEmbeddingTokens?: (texts: string[]) => Promise<number[]>
    }
  }

View on GitHub (pinned to fad3f12a14)

Solutions

1. Confirm the embedding model is fully loaded and countEmbeddingTokens works on a sample before bulk ingestion.
2. Restart the llamacpp extension and retry the batch.
3. Ensure tokenizer files ship with the embedding model.
4. If the method is genuinely unavailable, the caller skips verification only when the method is absent — so update/downgrade to a version that exposes it correctly.

Example fix

// before
await vecdbExt.ingestFile(threadId, file, opts)

// after
const llm = (window.core?.extensionManager.getByName('@janhq/llamacpp-extension') as any)
if (typeof llm?.countEmbeddingTokens === 'function') {
  await llm.countEmbeddingTokens(['health check']) // throws early if unhealthy
}
await vecdbExt.ingestFile(threadId, file, opts)

const llm = (window.core?.extensionManager.getByName('@janhq/llamacpp-extension') as any)
if (typeof llm?.countEmbeddingTokens === 'function') {
  const ok = await llm.countEmbeddingTokens(['health check']).then(() => true).catch(() => false)
  if (!ok) { /* embedding engine unhealthy; reload */ }
}

try {
  await vecdbExt.ingestFile(threadId, file, opts)
} catch (e) {
  if (e instanceof Error && e.message.startsWith('Failed to count embedding tokens')) {
    await reloadEmbeddingModel()
    return vecdbExt.ingestFile(threadId, file, opts)
  }
  throw e
}

When it happens

Trigger: ensureChunksFitEmbeddingContext splits a chunk and calls countEmbeddingTokens, which rejects because the embedding server is down, the model is mid-load, or the tokenizer is unavailable.

Common situations: Embedding model still loading during ingestion; llamacpp extension crashed mid-batch; tokenizer files missing from the model directory.

Related errors

• Failed to determine embedding context size: ${e instanceof E
• llamacpp extension not available
• llamacpp extension not available
• Embedding dimension not available
• Tokenize request failed with status ${res.status}

AI-assisted analysis of janhq/jan@fad3f12a14 (2026-08-12). Data as JSON: /api/errors/d15da87a21a2ddec. Report an issue: GitHub.