janhq/jan · error · Error

Failed to determine embedding context size

Error message

Failed to determine embedding context size: ${e instanceof Error ? e.message : String(e)}

What it means

probeEmbeddingContextSize wraps any failure from the embedding engine's getEmbeddingContextSize() call into a single descriptive error. The extension needs to know the model's max context size to size chunks; if the engine (local model, remote API, or plugin) fails to answer, this error propagates with the underlying message appended.

Solutions

Check the underlying message in the error text to identify the root cause (auth, network, model load).
Verify the embedding engine/provider configuration (API key, base URL, model name) is valid and reachable.
Confirm the configured EmbeddingEngine implements getEmbeddingContextSize() and that its model files/plugins are installed.
Test the embedding backend independently (e.g. a direct embed request) to confirm it is healthy before ingesting.

Example fix

// before
const size = await ctxSize(llm) // throws 'Failed to determine embedding context size: ...'
// after
let size: number | undefined
try {
  size = await ctxSize(llm)
} catch (e) {
  console.error('embedding engine unreachable:', e)
  size = undefined // fall back to a conservative default chunk size
}

Defensive patterns

Strategy: try-catch

Validate before calling

if (typeof llm.getEmbeddingContextSize !== 'function') {
  throw new Error('embedding engine does not implement getEmbeddingContextSize')
}

Type guard

function hasCtxSize(e: unknown): e is EmbeddingEngine & { getEmbeddingContextSize(): Promise<number> } {
  return typeof (e as any)?.getEmbeddingContextSize === 'function'
}

Try / catch

try {
  const size = await probeEmbeddingContextSize(llm)
} catch (e) {
  const cause = e instanceof Error ? e.message : String(e)
  if (/auth|api key|401|403/i.test(cause)) fixCredentials()
  else if (/network|fetch|timeout/i.test(cause)) await retryWithBackoff()
  else useDefaultChunkBudget()
}

Prevention

Validate embedding provider config (key, URL, model) at startup, before any ingest.
Health-check the embedding engine with a trivial embed call before large operations.
Provide a sensible default context size fallback for engines without the API.

When it happens

Trigger: Calling ctxSize() (which invokes probeEmbeddingContextSize) when the configured embedding engine throws from getEmbeddingContextSize() — e.g. the model/backend is unavailable, misconfigured, or the engine implementation rejects the call.

Common situations: Embedding provider API key missing or invalid, local embedding model failed to load, network outage when querying a remote embedding service, or an engine that does not implement getEmbeddingContextSize.

Related errors

AI-assisted analysis of janhq/jan@7205d770c1 (2026-09-17). Data as JSON: /api/errors/e43650a792215db1. Report an issue: GitHub.

Appendix: source

Thrown at extensions/vector-db-extension/src/index.ts:211

    for (const chunk of chunks) {
      out.push(...(await this.splitChunkToFit(chunk, budget, llm)))
    }
    return out
  }

  /**
   * A rejected probe/count means the embedding engine is unhealthy (e.g. the
   * embedding model failed to load). Skipping verification here would let
   * oversized chunks through and surface later as a confusing HTTP 400
   * (exceed_context_size_error), so fail ingestion with the real cause.
   */
  private async probeEmbeddingContextSize(
    llm: EmbeddingEngine
  ): Promise<number | undefined> {
    try {
      return await llm.getEmbeddingContextSize()
    } catch (e) {
      throw new Error(
        `Failed to determine embedding context size: ${e instanceof Error ? e.message : String(e)}`
      )
    }
  }

  private async splitChunkToFit(
    text: string,
    budget: number,
    llm: EmbeddingEngine
  ): Promise<string[]> {
    if (!text) return []
    let count: number
    try {
      ;[count] = await llm.countEmbeddingTokens([text])
    } catch (e) {
      throw new Error(
        `Failed to count embedding tokens: ${e instanceof Error ? e.message : String(e)}`
      )

View on GitHub (pinned to 7205d770c1)