janhq/jan · error

ubatch_size ( ) is too small. Minimum required

Error message

ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}

What it means

Validation guard in buildEmbedBatches (extensions/llamacpp-extension/src/util.ts:302): ubatch_size must be at least ceil(1/UBATCH_SAFETY_MARGIN) so a safety-margin-scaled limit still fits at least one token slot. A value below that minimum would produce empty or invalid batches, so the function refuses to run.

Solutions

1. Pass a ubatchSize >= Math.ceil(1/UBATCH_SAFETY_MARGIN) (compute it or use the documented minimum)
2. Read the actual ubatch_size from the llama.cpp server settings and sanitize 0/undefined to a default (e.g. 512)
3. Clamp: const size = Math.max(ubatchSize, minUbatchSize) before calling
4. Fix the source that produced 0 (device query, config parse) rather than the batch call

Example fix

// before
await buildEmbedBatches(inputs, settings.ubatch_size)
// after
const MIN = Math.ceil(1 / UBATCH_SAFETY_MARGIN)
const size = Math.max(Number(settings.ubatch_size) || 512, MIN)
await buildEmbedBatches(inputs, size)

const MIN = Math.ceil(1 / UBATCH_SAFETY_MARGIN)
const size = Number(settings.ubatch_size)
if (!Number.isFinite(size) || size < MIN) {
  throw new Error(`ubatch_size must be >= ${MIN}, got ${settings.ubatch_size}`)
}

When it happens

Trigger: Calling the batching/batches API (directly or via embeddings) with ubatchSize computed as 0 or a very small number — typically from misread server settings, division rounding, or passing 0 when ubatch config is absent.

Common situations: Config files where llama.cpp's ubatch_size was set to 0 or omitted and defaulted badly, or programmatic callers computing batch size from a device query that returned 0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Current backend not found
• llamacpp extension not available
• llamacpp extension not available
• Provider configuration is required
• Provider must have base_url configured

AI-assisted analysis of janhq/jan@7205d770c1 (2026-09-17). Data as JSON: /api/errors/bb35f01d21031def. Report an issue: GitHub.

Appendix: source

Thrown at extensions/llamacpp-extension/src/util.ts:302

export function truncateToTokenBudget(
  text: string,
  maxTokens: number,
  charsPerToken = DEFAULT_CHARS_PER_TOKEN
): string {
  const cpt = Math.max(charsPerToken, 1)
  const maxChars = Math.max(1, maxTokens) * cpt
  if (text.length <= maxChars) return text
  return text.slice(0, maxChars)
}

export function buildEmbedBatches(
  inputs: string[],
  ubatchSize: number,
  charsPerToken = DEFAULT_CHARS_PER_TOKEN
): EmbedBatch[] {
  const minUbatchSize = Math.ceil(1 / UBATCH_SAFETY_MARGIN)
  if (ubatchSize < minUbatchSize) {
    throw new Error(
      `ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}`
    )
  }

  const safeLimit = Math.floor(ubatchSize * UBATCH_SAFETY_MARGIN)

  const batches: EmbedBatch[] = []
  let current: string[] = []
  let currentTokens = 0
  let offset = 0

  const push = () => {
    if (current.length) {
      batches.push({ batch: current, offset })
      offset += current.length
      current = []
      currentTokens = 0
    }

View on GitHub (pinned to 7205d770c1)