janhq/jan · error · Error

ubatch_size (${ubatchSize}) is too small. Minimum required:

Error message

ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}

What it means

Thrown by buildEmbedBatches() when ubatchSize is below the floor. The floor is Math.ceil(1 / UBATCH_SAFETY_MARGIN) where UBATCH_SAFETY_MARGIN=0.5, so minUbatchSize=2. Because the safe token budget is ubatchSize*0.5, a ubatch of 1 would round down to 0 usable tokens and break batching entirely.

Source

Thrown at extensions/llamacpp-extension/src/util.ts:303

export function truncateToTokenBudget(
  text: string,
  maxTokens: number,
  charsPerToken = DEFAULT_CHARS_PER_TOKEN
): string {
  const cpt = Math.max(charsPerToken, 1)
  const maxChars = Math.max(1, maxTokens) * cpt
  if (text.length <= maxChars) return text
  return text.slice(0, maxChars)
}

export function buildEmbedBatches(
  inputs: string[],
  ubatchSize: number,
  charsPerToken = DEFAULT_CHARS_PER_TOKEN
): EmbedBatch[] {
  const minUbatchSize = Math.ceil(1 / UBATCH_SAFETY_MARGIN)
  if (ubatchSize < minUbatchSize) {
    throw new Error(
      `ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}`
    )
  }

  const safeLimit = Math.floor(ubatchSize * UBATCH_SAFETY_MARGIN)

  const batches: EmbedBatch[] = []
  let current: string[] = []
  let currentTokens = 0
  let offset = 0

  const push = () => {
    if (current.length) {
      batches.push({ batch: current, offset })
      offset += current.length
      current = []
      currentTokens = 0
    }

View on GitHub (pinned to fad3f12a14)

Solutions

1. Set ubatch_size in llamacpp settings to at least 2 (the default 512 is safe).
2. In custom callers, clamp ubatch to a sane minimum (e.g. Math.max(64, configuredUbatch)) before calling embed().
3. If memory is the concern, lower it modestly (e.g. 64-128), never to 1.

Example fix

// before
const batches = buildEmbedBatches(text, ubatchSize)

// after
const safeUbatch = ubatchSize && ubatchSize >= 2 ? ubatchSize : 512
const batches = buildEmbedBatches(text, safeUbatch)

const MIN_UBATCH = 2
const safeUbatch = Number.isFinite(ubatchSize) && ubatchSize >= MIN_UBATCH ? ubatchSize : 512
const batches = buildEmbedBatches(text, safeUbatch)

function isValidUbatch(n: unknown): n is number {
  return typeof n === 'number' && Number.isFinite(n) && n >= 2
}

When it happens

Trigger: engine.embed() is called with this.config.ubatch_size set to 0, 1, negative, or NaN; a custom code path passes buildEmbedBatches a manually computed ubatch below 2; settings UI let a user save ubatch_size=1.

Common situations: User lowered ubatch_size in advanced settings to reduce memory and entered 1; a config migration left ubatch_size as 0; embed() is called before config is loaded so ubatch_size is undefined and a bad fallback is used (though embed() defaults to 512, custom callers may not).

Related errors

• Invalid backend string: ${targetBackendString} supplied to u
• Invalid backend string format: "${targetBackendString}". Exp
• Invalid path or file ${path}
• Failed to parse archive name: ${archiveName}. Expected forma
• Invalid backend archive name: ${archiveName}

AI-assisted analysis of janhq/jan@fad3f12a14 (2026-08-12). Data as JSON: /api/errors/bb35f01d21031def. Report an issue: GitHub.