{"record":{"id":"6205b9c36a6e4c48","repo":"headroomlabs-ai/headroom","slug":"max-batch-bytes-must-be-at-least-min-batch-bytes","errorCode":null,"errorMessage":"max_batch_bytes must be at least min_batch_bytes","messagePattern":"max_batch_bytes must be at least min_batch_bytes","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"headroom/transforms/compression_batches.py","lineNumber":78,"sourceCode":"def build_compression_batches(\n    entries: list[CompressionBatchEntry],\n    *,\n    min_batch_bytes: int,\n    max_batch_bytes: int = DEFAULT_MAX_BATCH_BYTES,\n    max_batch_units: int = DEFAULT_MAX_BATCH_UNITS,\n) -> tuple[list[CompressionBatch], list[CompressionBatchEntry]]:\n    \"\"\"Greedily group compatible small units and skip under-floor tails.\n\n    Callers retain the skipped entries as normal ``size_floor`` results. The\n    function deliberately does not turn a unit larger than the configured\n    batch ceiling into a singleton batch; those units belong to the existing\n    independent compression path.\n    \"\"\"\n\n    if min_batch_bytes <= 0:\n        raise ValueError(\"min_batch_bytes must be positive\")\n    if max_batch_bytes < min_batch_bytes:\n        raise ValueError(\"max_batch_bytes must be at least min_batch_bytes\")\n    if max_batch_units <= 0:\n        raise ValueError(\"max_batch_units must be positive\")\n\n    batches: list[CompressionBatch] = []\n    skipped: list[CompressionBatchEntry] = []\n    pending: list[CompressionBatchEntry] = []\n    pending_bytes = 0\n    pending_key: tuple[object, ...] | None = None\n\n    def flush() -> None:\n        nonlocal pending, pending_bytes, pending_key\n        if not pending:\n            return\n        if pending_bytes >= min_batch_bytes:\n            batches.append(CompressionBatch(entries=tuple(pending), text_bytes=pending_bytes))\n        else:\n            skipped.extend(pending)\n        pending = []","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/transforms/compression_batches.py#L60-L96","documentation":"Argument-validation ValueError from the batch-grouping function: max_batch_bytes < min_batch_bytes. The invariant matters because the greedy accumulator flushes when pending_bytes reaches the max; a ceiling below the floor means no batch could ever satisfy both constraints, so the function rejects the pair up front.","triggerScenarios":"Calling the function with max_batch_bytes smaller than min_batch_bytes, e.g. min=4096/max=2048 — usually two independently configured knobs that drifted apart.","commonSituations":"Env vars or YAML tuned by different people at different times; lowering max_batch_bytes for memory reasons without re-checking min_batch_bytes; defaults overridden per-environment inconsistently.","solutions":["Ensure max_batch_bytes >= min_batch_bytes (e.g. min=2048, max=32768).","Add a config sanity check at startup: if not (0 < min <= max): fail with a clear message.","Derive one from the other (max = 8*min) if you only want one knob."],"exampleFix":"# before\ngroup_batches(entries, min_batch_bytes=8192, max_batch_bytes=4096)\n\n# after\ngroup_batches(entries, min_batch_bytes=2048, max_batch_bytes=max(2048, cfg.max_batch_bytes))","handlingStrategy":"validation","validationCode":"if not (0 < min_batch_bytes <= max_batch_bytes):\n    raise ConfigError(f\"batch bytes misconfigured: min={min_batch_bytes}, max={max_batch_bytes}\")","typeGuard":null,"tryCatchPattern":"try:\n    group_batches(entries, min_batch_bytes=a, max_batch_bytes=b)\nexcept ValueError as e:\n    if \"at least min_batch_bytes\" in str(e):\n        group_batches(entries, min_batch_bytes=a, max_batch_bytes=max(a, b))\n    else:\n        raise","preventionTips":["Treat min/max as one coupled setting; change them together.","Derive max from min (e.g. max = 8*min) to make the invariant structural.","Assert the invariant in a config sanity test."],"tags":["validation","configuration","batching"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}