{"record":{"id":"28dc5264d313bec4","repo":"xai-org/x-algorithm","slug":"eval-bs-per-device-eval-bs-train-bs-per-device","errorCode":null,"errorMessage":"eval_bs_per_device={eval_bs} > train bs_per_device={self.bs_per_device}; eval B must fit in the train batch (slice prefix) or lower train B","messagePattern":"eval_bs_per_device=(.+?) > train bs_per_device=(.+?); eval B must fit in the train batch \\(slice prefix\\) or lower train B","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/train/trainer_sid_retrieval.py","lineNumber":113,"sourceCode":"        from xrex.eval.recsys_eval import (\n            finalize_sid_eval_metrics,\n            log_sid_eval_summary,\n            make_sid_retrieval_eval_task,\n            sid_eval_batch_geometry,\n        )\n\n        assert isinstance(self.model_config, RecsysSIDRetrievalConfig)\n        assert isinstance(self.state, RecsysTrainingState)\n\n        rank_logger.info(\"Running SID retrieval eval at step %s\", soft_step)\n        t0 = time.time()\n        rng = jax.random.PRNGKey(soft_step)\n        beam_width = max(1, int(self.model_config.eval_beam_width))\n        eval_bs, eval_global_b, host_local_b = sid_eval_batch_geometry(\n            int(self.model_config.eval_bs_per_device), self.num_devices\n        )\n        if eval_bs > self.bs_per_device:\n            raise ValueError(\n                f\"eval_bs_per_device={eval_bs} > train bs_per_device={self.bs_per_device}; \"\n                \"eval B must fit in the train batch (slice prefix) or lower train B\"\n            )\n\n        if beam_width not in self._sid_beam_generate_jits:\n            fn = self._make_sid_generate_fn(beam_width)\n            self._sid_beam_generate_fns[beam_width] = fn\n            self._sid_beam_generate_jits[beam_width] = jax.jit(fn.apply)\n        beam_jit = self._sid_beam_generate_jits[beam_width]\n\n        saved_batch_size = int(self.batch_size)\n\n        def beam_forward_fn(jax_batch):\n            emb = self.get_recsys_embeddings(jax_batch, self.state.emb_table)\n            return beam_jit(self.state.params, rng, jax_batch, emb)\n\n        try:\n            object.__setattr__(self, \"batch_size\", eval_global_b)","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/train/trainer_sid_retrieval.py#L95-L131","documentation":"The SID retrieval evaluator slices an eval batch as a prefix of the training batch, so the per-device eval batch size must not exceed the per-device training batch size. This check fails fast before JIT-ing when eval_bs_per_device (adjusted for device count by sid_eval_batch_geometry) exceeds the trainer's bs_per_device.","triggerScenarios":"Running eval (which calls eval_sid_retrieval) with model_config.eval_bs_per_device larger than the training bs_per_device, or where the batch-geometry helper rounds eval_bs up across num_devices so it exceeds the train value.","commonSituations":"Reusing an eval-heavy config on a smaller train batch; lowering train batch size for memory reasons without lowering eval batch; multi-device runs where geometry math changes effective per-device size.","solutions":["Set eval_bs_per_device <= bs_per_device in the model config","If larger eval batches are needed, increase train bs_per_device (may require more memory)","Verify num_devices matches the actual topology so sid_eval_batch_geometry computes the intended per-device size"],"exampleFix":"# before\nmodel_config.eval_bs_per_device = 64\nmodel_config.bs_per_device = 32\n# after\nmodel_config.eval_bs_per_device = 32\nmodel_config.bs_per_device = 32","handlingStrategy":"validation","validationCode":"if model_config.eval_bs_per_device > self.bs_per_device:\n    model_config.eval_bs_per_device = self.bs_per_device  # clamp before eval()","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive eval_bs_per_device from bs_per_device in config generation (e.g. min(eval, train))","Add a config sanity assert: eval_bs_per_device <= bs_per_device","Re-check after any train-batch-size reduction"],"tags":["batch-size","eval","config-validation","jax"],"backgroundTag":"batch-size-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}