xai-org/x-algorithm · error · ValueError

eval_bs_per_device={eval_bs} > train bs_per_device={self.bs_

Error message

eval_bs_per_device={eval_bs} > train bs_per_device={self.bs_per_device}; eval B must fit in the train batch (slice prefix) or lower train B

What it means

The SID retrieval evaluator slices an eval batch as a prefix of the training batch, so the per-device eval batch size must not exceed the per-device training batch size. This check fails fast before JIT-ing when eval_bs_per_device (adjusted for device count by sid_eval_batch_geometry) exceeds the trainer's bs_per_device.

Source

Thrown at phoenix/xrex/train/trainer_sid_retrieval.py:113

        from xrex.eval.recsys_eval import (
            finalize_sid_eval_metrics,
            log_sid_eval_summary,
            make_sid_retrieval_eval_task,
            sid_eval_batch_geometry,
        )

        assert isinstance(self.model_config, RecsysSIDRetrievalConfig)
        assert isinstance(self.state, RecsysTrainingState)

        rank_logger.info("Running SID retrieval eval at step %s", soft_step)
        t0 = time.time()
        rng = jax.random.PRNGKey(soft_step)
        beam_width = max(1, int(self.model_config.eval_beam_width))
        eval_bs, eval_global_b, host_local_b = sid_eval_batch_geometry(
            int(self.model_config.eval_bs_per_device), self.num_devices
        )
        if eval_bs > self.bs_per_device:
            raise ValueError(
                f"eval_bs_per_device={eval_bs} > train bs_per_device={self.bs_per_device}; "
                "eval B must fit in the train batch (slice prefix) or lower train B"
            )

        if beam_width not in self._sid_beam_generate_jits:
            fn = self._make_sid_generate_fn(beam_width)
            self._sid_beam_generate_fns[beam_width] = fn
            self._sid_beam_generate_jits[beam_width] = jax.jit(fn.apply)
        beam_jit = self._sid_beam_generate_jits[beam_width]

        saved_batch_size = int(self.batch_size)

        def beam_forward_fn(jax_batch):
            emb = self.get_recsys_embeddings(jax_batch, self.state.emb_table)
            return beam_jit(self.state.params, rng, jax_batch, emb)

        try:
            object.__setattr__(self, "batch_size", eval_global_b)

View on GitHub (pinned to 24c60942c5)

Solutions

Set eval_bs_per_device <= bs_per_device in the model config
If larger eval batches are needed, increase train bs_per_device (may require more memory)
Verify num_devices matches the actual topology so sid_eval_batch_geometry computes the intended per-device size

Example fix

# before
model_config.eval_bs_per_device = 64
model_config.bs_per_device = 32
# after
model_config.eval_bs_per_device = 32
model_config.bs_per_device = 32

Defensive patterns

Strategy: validation

Validate before calling

if model_config.eval_bs_per_device > self.bs_per_device:
    model_config.eval_bs_per_device = self.bs_per_device  # clamp before eval()

Prevention

Derive eval_bs_per_device from bs_per_device in config generation (e.g. min(eval, train))
Add a config sanity assert: eval_bs_per_device <= bs_per_device
Re-check after any train-batch-size reduction

When it happens

Trigger: Running eval (which calls eval_sid_retrieval) with model_config.eval_bs_per_device larger than the training bs_per_device, or where the batch-geometry helper rounds eval_bs up across num_devices so it exceeds the train value.

Common situations: Reusing an eval-heavy config on a smaller train batch; lowering train batch size for memory reasons without lowering eval batch; multi-device runs where geometry math changes effective per-device size.

Related errors

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/28dc5264d313bec4. Report an issue: GitHub.