lancedb/lancedb · error · ValueError

blocks_per_epoch must be a positive integer or 'auto'

Error message

blocks_per_epoch must be a positive integer or 'auto'

What it means

blocks_per_epoch must be either the string "auto" or a true integer (bool is explicitly rejected because it is an int subclass in Python). Anything else — a float like 1024.0, a numeric string like "1024", or True/False — raises ValueError.

Solutions

1. Pass a real Python int: blocks_per_epoch=int(config_value), or the literal string "auto".
2. Sanitize loaded configs: if v != 'auto': v = int(float(v)) — and reject bools.
3. Fix the config file so the value is an unquoted integer or the exact token auto.

Example fix

// before
blocks_per_epoch = cfg["blocks_per_epoch"]  # 1024.0 (float from YAML)
StreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)
// after
bpe = cfg["blocks_per_epoch"]
blocks_per_epoch = "auto" if bpe == "auto" else int(bpe)
StreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)

def coerce_blocks_per_epoch(v):
    if v == "auto":
        return "auto"
    if isinstance(v, bool) or not isinstance(v, int):
        raise ValueError("blocks_per_epoch must be a positive int or 'auto'")
    return v

def is_valid_blocks_per_epoch(v) -> bool:
    return v == "auto" or (isinstance(v, int) and not isinstance(v, bool))

try:
    dataset = StreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)
except ValueError as e:
    if "positive integer or 'auto'" in str(e):
        blocks_per_epoch = "auto"
        dataset = StreamingTrainDataset(..., blocks_per_epoch=blocks_per_epoch)
    else:
        raise

When it happens

Trigger: StreamingTrainDataset(..., pack_sequences=2048, eos_id=..., pad_id=..., blocks_per_epoch=1024.0) (float), blocks_per_epoch="1024" (str), or blocks_per_epoch=True; JSON/YAML configs often load these as float/string.

Common situations: Config files where all numbers parse as floats (YAML '1024' -> float in some parsers) or strings; CLI arguments parsed with float(); a boolean flag accidentally bound to the blocks_per_epoch key.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• pack_sequences must be greater than 0
• a Function environment is pip or conda, not both
• All elements in vector_results should be of the same type
• All elements in vector_results should be of the same type
• All elements in vector_results should be of the same type

AI-assisted analysis of lancedb/lancedb@c7b051aff7 (2026-09-08). Data as JSON: /api/errors/4f33c8ecbbc81594. Report an issue: GitHub.

Appendix: source

Thrown at python/python/lancedb/streaming.py:448

            raise ValueError("io_queue_depth must be greater than 0")
        if transform_parallelism is not None and transform_parallelism <= 0:
            raise ValueError("transform_parallelism must be greater than 0")
        if pack_sequences is not None:
            if pack_sequences <= 0:
                raise ValueError("pack_sequences must be greater than 0")
            if eos_id is None:
                raise ValueError("eos_id is required when pack_sequences is set")
            if pad_id is None:
                raise ValueError("pad_id is required when pack_sequences is set")
            if blocks_per_epoch is None:
                raise ValueError(
                    "blocks_per_epoch is required when pack_sequences is set"
                )
            if blocks_per_epoch != "auto":
                if not isinstance(blocks_per_epoch, int) or isinstance(
                    blocks_per_epoch, bool
                ):
                    raise ValueError(
                        "blocks_per_epoch must be a positive integer or 'auto'"
                    )
                if blocks_per_epoch <= 0:
                    raise ValueError("blocks_per_epoch must be greater than 0")
                if blocks_per_epoch % num_splits != 0:
                    raise ValueError(
                        f"blocks_per_epoch ({blocks_per_epoch}) must be divisible by "
                        f"num_splits ({num_splits})"
                    )
            if transform is not None:
                raise ValueError("transform cannot be combined with pack_sequences")
            if columns is None or len(columns) != 1:
                raise ValueError(
                    "pack_sequences requires columns to name exactly one "
                    "list-typed column of token ids"
                )
            field = table.schema.field(columns[0])
            if not (

View on GitHub (pinned to c7b051aff7)