JuliusBrussee/caveman · error · Error

invalid chars-per-token ratio

Error message

invalid chars-per-token ratio: ${ratio}

What it means

estimateTokens() converts a character count to an approximate token count using a chars-per-token ratio that must be strictly positive (the library calibrates 5.9–6.9 against provider-exact Claude Code prefixes; default 6.4). A ratio of 0, negative, NaN, or a non-numeric value would produce Infinity/NaN token counts, so it is rejected with this error.

Solutions

1. Pass a positive number for ratio (omit the argument to use DEFAULT_CHARS_PER_TOKEN = 6.4).
2. Coerce config values with Number(value) and check Number.isFinite(ratio) && ratio > 0 before calling.
3. Fix the upstream computation producing NaN/0 (e.g. guard division by zero when deriving a measured chars/token ratio).
4. Check argument order — the signature is estimateTokens(chars, ratio); ensure you are not passing a char count as the ratio.

Example fix

// before
const ratio = Number(process.env.CHARS_PER_TOKEN); // NaN when unset
estimateTokens(chars, ratio); // throws
// after
const raw = Number(process.env.CHARS_PER_TOKEN);
const ratio = Number.isFinite(raw) && raw > 0 ? raw : DEFAULT_CHARS_PER_TOKEN;
estimateTokens(chars, ratio);

function validRatio(v) {
  const n = typeof v === "number" ? v : Number(v);
  return Number.isFinite(n) && n > 0 ? n : DEFAULT_CHARS_PER_TOKEN;
}
estimateTokens(chars, validRatio(configuredRatio));

const isPositiveNumber = (v) => typeof v === "number" && Number.isFinite(v) && v > 0;

try {
  return estimateTokens(chars, ratio);
} catch (e) {
  if (e.message.startsWith("invalid chars-per-token ratio")) {
    return estimateTokens(chars, DEFAULT_CHARS_PER_TOKEN); // fall back to calibrated default
  } else throw e;
}

When it happens

Trigger: Calling estimateTokens(chars, ratio) with ratio = 0, a negative number, NaN (e.g. from a failed parseFloat of config), undefined chained wrongly, or a string like "6.4" that fails the > 0 numeric coercion check.

Common situations: Loading the ratio from config/env where the value failed to parse into a number; dividing to compute a measured ratio and getting 0 or NaN when the denominator was 0; typo passing chars and ratio arguments swapped.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• aes cipher
• apiKey, baseURL, and agent are required
• ASGI context resolver or request bounds are invalid
• AutoGen tools and workbench are mutually exclusive
• awssig: signer requires region and service

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/789469b8371f34d3. Report an issue: GitHub.

Appendix: source

Thrown at packages/subagent-tax/lib/tokens.mjs:15

// Token accounting. Two rungs, never blended:
//  - "est":   chars / ratio. Calibrated against provider-exact Claude Code runs
//             on 2026-08-07 (observed 5.9–6.9 chars/token on full harness
//             prefixes; 6.4 is the midpoint). One machine's calibration — a
//             labeled estimate, not a measurement.
//  - "exact": Anthropic's count_tokens endpoint (a free, separate rate bucket),
//             opt-in via --count-tokens + ANTHROPIC_API_KEY, and only for
//             anthropic-protocol captures (other tokenizers differ).

export const DEFAULT_CHARS_PER_TOKEN = 6.4;
export const CALIBRATION_NOTE =
  "chars/token calibrated 5.9–6.9 against provider-exact Claude Code prefixes (2026-08-07, one machine); 6.4 = midpoint";

export function estimateTokens(chars, ratio = DEFAULT_CHARS_PER_TOKEN) {
  if (!(ratio > 0)) throw new Error(`invalid chars-per-token ratio: ${ratio}`);
  return { tokens: Math.round(chars / ratio), basis: "est", ratio };
}

// Rebuild a count_tokens request from a captured anthropic-messages body,
// passing the captured fields through verbatim so the count is of what the
// harness actually sent.
export function buildCountTokensRequest(capturedBody) {
  const { model, system, tools, messages } = capturedBody;
  if (!model || !Array.isArray(messages)) return null;
  const req = { model, messages };
  if (system !== undefined) req.system = system;
  if (Array.isArray(tools) && tools.length > 0) req.tools = tools;
  return req;
}

export async function countTokensAnthropic(capturedBody, { apiKey, baseUrl = "https://api.anthropic.com", fetchImpl = fetch } = {}) {
  const req = buildCountTokensRequest(capturedBody);
  if (!req) return { error: "capture is not a countable anthropic-messages body" };

View on GitHub (pinned to 3ee70a1026)