{"record":{"id":"4f33c8ecbbc81594","repo":"lancedb/lancedb","slug":"blocks-per-epoch-must-be-a-positive-integer-or-au","errorCode":null,"errorMessage":"blocks_per_epoch must be a positive integer or 'auto'","messagePattern":"blocks_per_epoch must be a positive integer or 'auto'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/python/lancedb/streaming.py","lineNumber":448,"sourceCode":"            raise ValueError(\"io_queue_depth must be greater than 0\")\n        if transform_parallelism is not None and transform_parallelism <= 0:\n            raise ValueError(\"transform_parallelism must be greater than 0\")\n        if pack_sequences is not None:\n            if pack_sequences <= 0:\n                raise ValueError(\"pack_sequences must be greater than 0\")\n            if eos_id is None:\n                raise ValueError(\"eos_id is required when pack_sequences is set\")\n            if pad_id is None:\n                raise ValueError(\"pad_id is required when pack_sequences is set\")\n            if blocks_per_epoch is None:\n                raise ValueError(\n                    \"blocks_per_epoch is required when pack_sequences is set\"\n                )\n            if blocks_per_epoch != \"auto\":\n                if not isinstance(blocks_per_epoch, int) or isinstance(\n                    blocks_per_epoch, bool\n                ):\n                    raise ValueError(\n                        \"blocks_per_epoch must be a positive integer or 'auto'\"\n                    )\n                if blocks_per_epoch <= 0:\n                    raise ValueError(\"blocks_per_epoch must be greater than 0\")\n                if blocks_per_epoch % num_splits != 0:\n                    raise ValueError(\n                        f\"blocks_per_epoch ({blocks_per_epoch}) must be divisible by \"\n                        f\"num_splits ({num_splits})\"\n                    )\n            if transform is not None:\n                raise ValueError(\"transform cannot be combined with pack_sequences\")\n            if columns is None or len(columns) != 1:\n                raise ValueError(\n                    \"pack_sequences requires columns to name exactly one \"\n                    \"list-typed column of token ids\"\n                )\n            field = table.schema.field(columns[0])\n            if not (","sourceCodeStart":430,"sourceCodeEnd":466,"githubUrl":"https://github.com/lancedb/lancedb/blob/c7b051aff7039333a3f61b79217246c27676806a/python/python/lancedb/streaming.py#L430-L466","documentation":"blocks_per_epoch must be either the string \"auto\" or a true integer (bool is explicitly rejected because it is an int subclass in Python). Anything else — a float like 1024.0, a numeric string like \"1024\", or True/False — raises ValueError.","triggerScenarios":"StreamingTrainDataset(..., pack_sequences=2048, eos_id=..., pad_id=..., blocks_per_epoch=1024.0) (float), blocks_per_epoch=\"1024\" (str), or blocks_per_epoch=True; JSON/YAML configs often load these as float/string.","commonSituations":"Config files where all numbers parse as floats (YAML '1024' -> float in some parsers) or strings; CLI arguments parsed with float(); a boolean flag accidentally bound to the blocks_per_epoch key.","solutions":["Pass a real Python int: blocks_per_epoch=int(config_value), or the literal string \"auto\".","Sanitize loaded configs: if v != 'auto': v = int(float(v)) — and reject bools.","Fix the config file so the value is an unquoted integer or the exact token auto."],"exampleFix":"// before\nblocks_per_epoch = cfg[\"blocks_per_epoch\"]  # 1024.0 (float from YAML)\nStreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)\n// after\nbpe = cfg[\"blocks_per_epoch\"]\nblocks_per_epoch = \"auto\" if bpe == \"auto\" else int(bpe)\nStreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)","handlingStrategy":"type-guard","validationCode":"def coerce_blocks_per_epoch(v):\n    if v == \"auto\":\n        return \"auto\"\n    if isinstance(v, bool) or not isinstance(v, int):\n        raise ValueError(\"blocks_per_epoch must be a positive int or 'auto'\")\n    return v","typeGuard":"def is_valid_blocks_per_epoch(v) -> bool:\n    return v == \"auto\" or (isinstance(v, int) and not isinstance(v, bool))","tryCatchPattern":"try:\n    dataset = StreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)\nexcept ValueError as e:\n    if \"positive integer or 'auto'\" in str(e):\n        blocks_per_epoch = \"auto\"\n        dataset = StreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)\n    else:\n        raise","preventionTips":["Coerce config values with int() (and reject bools) before passing them to the dataset.","Quote check: YAML/JSON may load integers as floats or strings — validate types after loading.","Keep bool flags out of numeric config keys in your schema."],"tags":["python","type-mismatch","sequence-packing","configuration"],"backgroundTag":"type-mismatch","analyzedSha":"c7b051aff7039333a3f61b79217246c27676806a","analyzedAt":"2026-09-08T23:42:37.579Z","contentChangedAt":"2026-09-08T23:42:37.579Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}