{"record":{"id":"bb35f01d21031def","repo":"janhq/jan","slug":"ubatch-size-ubatchsize-is-too-small-minimum","errorCode":null,"errorMessage":"ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}","messagePattern":"ubatch_size \\((.+?)\\) is too small\\. Minimum required: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"extensions/llamacpp-extension/src/util.ts","lineNumber":302,"sourceCode":"export function truncateToTokenBudget(\n  text: string,\n  maxTokens: number,\n  charsPerToken = DEFAULT_CHARS_PER_TOKEN\n): string {\n  const cpt = Math.max(charsPerToken, 1)\n  const maxChars = Math.max(1, maxTokens) * cpt\n  if (text.length <= maxChars) return text\n  return text.slice(0, maxChars)\n}\n\nexport function buildEmbedBatches(\n  inputs: string[],\n  ubatchSize: number,\n  charsPerToken = DEFAULT_CHARS_PER_TOKEN\n): EmbedBatch[] {\n  const minUbatchSize = Math.ceil(1 / UBATCH_SAFETY_MARGIN)\n  if (ubatchSize < minUbatchSize) {\n    throw new Error(\n      `ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}`\n    )\n  }\n\n  const safeLimit = Math.floor(ubatchSize * UBATCH_SAFETY_MARGIN)\n\n  const batches: EmbedBatch[] = []\n  let current: string[] = []\n  let currentTokens = 0\n  let offset = 0\n\n  const push = () => {\n    if (current.length) {\n      batches.push({ batch: current, offset })\n      offset += current.length\n      current = []\n      currentTokens = 0\n    }","sourceCodeStart":284,"sourceCodeEnd":320,"githubUrl":"https://github.com/janhq/jan/blob/7205d770c1e097c3daf35a911176410e93bc5564/extensions/llamacpp-extension/src/util.ts#L284-L320","documentation":"Validation guard in buildEmbedBatches (extensions/llamacpp-extension/src/util.ts:302): ubatch_size must be at least ceil(1/UBATCH_SAFETY_MARGIN) so a safety-margin-scaled limit still fits at least one token slot. A value below that minimum would produce empty or invalid batches, so the function refuses to run.","triggerScenarios":"Calling the batching/batches API (directly or via embeddings) with ubatchSize computed as 0 or a very small number — typically from misread server settings, division rounding, or passing 0 when ubatch config is absent.","commonSituations":"Config files where llama.cpp's ubatch_size was set to 0 or omitted and defaulted badly, or programmatic callers computing batch size from a device query that returned 0.","solutions":["Pass a ubatchSize >= Math.ceil(1/UBATCH_SAFETY_MARGIN) (compute it or use the documented minimum)","Read the actual ubatch_size from the llama.cpp server settings and sanitize 0/undefined to a default (e.g. 512)","Clamp: const size = Math.max(ubatchSize, minUbatchSize) before calling","Fix the source that produced 0 (device query, config parse) rather than the batch call"],"exampleFix":"// before\nawait buildEmbedBatches(inputs, settings.ubatch_size)\n// after\nconst MIN = Math.ceil(1 / UBATCH_SAFETY_MARGIN)\nconst size = Math.max(Number(settings.ubatch_size) || 512, MIN)\nawait buildEmbedBatches(inputs, size)","handlingStrategy":"validation","validationCode":"const MIN = Math.ceil(1 / UBATCH_SAFETY_MARGIN)\nconst size = Number(settings.ubatch_size)\nif (!Number.isFinite(size) || size < MIN) {\n  throw new Error(`ubatch_size must be >= ${MIN}, got ${settings.ubatch_size}`)\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Sanitize ubatch_size from config: reject or default 0/undefined","Clamp to the documented minimum before calling batch APIs","Verify device-query results that feed ubatch computation never return 0","Add a unit test for buildEmbedBatches with boundary ubatch values"],"tags":["validation","embeddings","configuration","llamacpp"],"backgroundTag":"value-out-of-range","analyzedSha":"7205d770c1e097c3daf35a911176410e93bc5564","analyzedAt":"2026-09-17T14:27:30.100Z","contentChangedAt":"2026-09-17T14:27:30.100Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}