{"record":{"id":"bb35f01d21031def","repo":"janhq/jan","slug":"ubatch-size-ubatchsize-is-too-small-minimum","errorCode":null,"errorMessage":"ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}","messagePattern":"ubatch_size \\((.+?)\\) is too small\\. Minimum required: (.+?)","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"extensions/llamacpp-extension/src/util.ts","lineNumber":303,"sourceCode":"export function truncateToTokenBudget(\n  text: string,\n  maxTokens: number,\n  charsPerToken = DEFAULT_CHARS_PER_TOKEN\n): string {\n  const cpt = Math.max(charsPerToken, 1)\n  const maxChars = Math.max(1, maxTokens) * cpt\n  if (text.length <= maxChars) return text\n  return text.slice(0, maxChars)\n}\n\nexport function buildEmbedBatches(\n  inputs: string[],\n  ubatchSize: number,\n  charsPerToken = DEFAULT_CHARS_PER_TOKEN\n): EmbedBatch[] {\n  const minUbatchSize = Math.ceil(1 / UBATCH_SAFETY_MARGIN)\n  if (ubatchSize < minUbatchSize) {\n    throw new Error(\n      `ubatch_size (${ubatchSize}) is too small. Minimum required: ${minUbatchSize}`\n    )\n  }\n\n  const safeLimit = Math.floor(ubatchSize * UBATCH_SAFETY_MARGIN)\n\n  const batches: EmbedBatch[] = []\n  let current: string[] = []\n  let currentTokens = 0\n  let offset = 0\n\n  const push = () => {\n    if (current.length) {\n      batches.push({ batch: current, offset })\n      offset += current.length\n      current = []\n      currentTokens = 0\n    }","sourceCodeStart":285,"sourceCodeEnd":321,"githubUrl":"https://github.com/janhq/jan/blob/fad3f12a147d138388a66f0d92a02b2675f65294/extensions/llamacpp-extension/src/util.ts#L285-L321","documentation":"Thrown by buildEmbedBatches() when ubatchSize is below the floor. The floor is Math.ceil(1 / UBATCH_SAFETY_MARGIN) where UBATCH_SAFETY_MARGIN=0.5, so minUbatchSize=2. Because the safe token budget is ubatchSize*0.5, a ubatch of 1 would round down to 0 usable tokens and break batching entirely.","triggerScenarios":"engine.embed() is called with this.config.ubatch_size set to 0, 1, negative, or NaN; a custom code path passes buildEmbedBatches a manually computed ubatch below 2; settings UI let a user save ubatch_size=1.","commonSituations":"User lowered ubatch_size in advanced settings to reduce memory and entered 1; a config migration left ubatch_size as 0; embed() is called before config is loaded so ubatch_size is undefined and a bad fallback is used (though embed() defaults to 512, custom callers may not).","solutions":["Set ubatch_size in llamacpp settings to at least 2 (the default 512 is safe).","In custom callers, clamp ubatch to a sane minimum (e.g. Math.max(64, configuredUbatch)) before calling embed().","If memory is the concern, lower it modestly (e.g. 64-128), never to 1."],"exampleFix":"// before\nconst batches = buildEmbedBatches(text, ubatchSize)\n\n// after\nconst safeUbatch = ubatchSize && ubatchSize >= 2 ? ubatchSize : 512\nconst batches = buildEmbedBatches(text, safeUbatch)","handlingStrategy":"validation","validationCode":"const MIN_UBATCH = 2\nconst safeUbatch = Number.isFinite(ubatchSize) && ubatchSize >= MIN_UBATCH ? ubatchSize : 512\nconst batches = buildEmbedBatches(text, safeUbatch)","typeGuard":"function isValidUbatch(n: unknown): n is number {\n  return typeof n === 'number' && Number.isFinite(n) && n >= 2\n}","tryCatchPattern":null,"preventionTips":["Clamp ubatch_size to at least 2 (realistically >= 64) before calling embed().","Do not let the settings UI persist ubatch_size < 2.","Default to 512 when config is missing or zero."],"tags":["validation","embedding","config","batching","typescript"],"backgroundTag":null,"analyzedSha":"fad3f12a147d138388a66f0d92a02b2675f65294","analyzedAt":"2026-08-12T20:33:47.516Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}