{"record":{"id":"789469b8371f34d3","repo":"JuliusBrussee/caveman","slug":"invalid-chars-per-token-ratio-ratio","errorCode":null,"errorMessage":"invalid chars-per-token ratio: ${ratio}","messagePattern":"invalid chars-per-token ratio: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/subagent-tax/lib/tokens.mjs","lineNumber":15,"sourceCode":"// Token accounting. Two rungs, never blended:\n//  - \"est\":   chars / ratio. Calibrated against provider-exact Claude Code runs\n//             on 2026-08-07 (observed 5.9–6.9 chars/token on full harness\n//             prefixes; 6.4 is the midpoint). One machine's calibration — a\n//             labeled estimate, not a measurement.\n//  - \"exact\": Anthropic's count_tokens endpoint (a free, separate rate bucket),\n//             opt-in via --count-tokens + ANTHROPIC_API_KEY, and only for\n//             anthropic-protocol captures (other tokenizers differ).\n\nexport const DEFAULT_CHARS_PER_TOKEN = 6.4;\nexport const CALIBRATION_NOTE =\n  \"chars/token calibrated 5.9–6.9 against provider-exact Claude Code prefixes (2026-08-07, one machine); 6.4 = midpoint\";\n\nexport function estimateTokens(chars, ratio = DEFAULT_CHARS_PER_TOKEN) {\n  if (!(ratio > 0)) throw new Error(`invalid chars-per-token ratio: ${ratio}`);\n  return { tokens: Math.round(chars / ratio), basis: \"est\", ratio };\n}\n\n// Rebuild a count_tokens request from a captured anthropic-messages body,\n// passing the captured fields through verbatim so the count is of what the\n// harness actually sent.\nexport function buildCountTokensRequest(capturedBody) {\n  const { model, system, tools, messages } = capturedBody;\n  if (!model || !Array.isArray(messages)) return null;\n  const req = { model, messages };\n  if (system !== undefined) req.system = system;\n  if (Array.isArray(tools) && tools.length > 0) req.tools = tools;\n  return req;\n}\n\nexport async function countTokensAnthropic(capturedBody, { apiKey, baseUrl = \"https://api.anthropic.com\", fetchImpl = fetch } = {}) {\n  const req = buildCountTokensRequest(capturedBody);\n  if (!req) return { error: \"capture is not a countable anthropic-messages body\" };","sourceCodeStart":1,"sourceCodeEnd":33,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/packages/subagent-tax/lib/tokens.mjs#L1-L33","documentation":"estimateTokens() converts a character count to an approximate token count using a chars-per-token ratio that must be strictly positive (the library calibrates 5.9–6.9 against provider-exact Claude Code prefixes; default 6.4). A ratio of 0, negative, NaN, or a non-numeric value would produce Infinity/NaN token counts, so it is rejected with this error.","triggerScenarios":"Calling estimateTokens(chars, ratio) with ratio = 0, a negative number, NaN (e.g. from a failed parseFloat of config), undefined chained wrongly, or a string like \"6.4\" that fails the > 0 numeric coercion check.","commonSituations":"Loading the ratio from config/env where the value failed to parse into a number; dividing to compute a measured ratio and getting 0 or NaN when the denominator was 0; typo passing chars and ratio arguments swapped.","solutions":["Pass a positive number for ratio (omit the argument to use DEFAULT_CHARS_PER_TOKEN = 6.4).","Coerce config values with Number(value) and check Number.isFinite(ratio) && ratio > 0 before calling.","Fix the upstream computation producing NaN/0 (e.g. guard division by zero when deriving a measured chars/token ratio).","Check argument order — the signature is estimateTokens(chars, ratio); ensure you are not passing a char count as the ratio."],"exampleFix":"// before\nconst ratio = Number(process.env.CHARS_PER_TOKEN); // NaN when unset\nestimateTokens(chars, ratio); // throws\n// after\nconst raw = Number(process.env.CHARS_PER_TOKEN);\nconst ratio = Number.isFinite(raw) && raw > 0 ? raw : DEFAULT_CHARS_PER_TOKEN;\nestimateTokens(chars, ratio);","handlingStrategy":"validation","validationCode":"function validRatio(v) {\n  const n = typeof v === \"number\" ? v : Number(v);\n  return Number.isFinite(n) && n > 0 ? n : DEFAULT_CHARS_PER_TOKEN;\n}\nestimateTokens(chars, validRatio(configuredRatio));","typeGuard":"const isPositiveNumber = (v) => typeof v === \"number\" && Number.isFinite(v) && v > 0;","tryCatchPattern":"try {\n  return estimateTokens(chars, ratio);\n} catch (e) {\n  if (e.message.startsWith(\"invalid chars-per-token ratio\")) {\n    return estimateTokens(chars, DEFAULT_CHARS_PER_TOKEN); // fall back to calibrated default\n  } else throw e;\n}","preventionTips":["Validate env/config numeric values with Number() and Number.isFinite before use.","Never pass a computed ratio without guarding the division that produced it (avoid 0/NaN).","Keep the ratio within the calibrated 5.9–6.9 band; omit the argument to use the 6.4 default."],"tags":["invalid-argument-value","configuration","tokenization"],"backgroundTag":"invalid-argument-value","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}