{"record":{"id":"2eb14693a52c50b6","repo":"microsoft/qlib","slug":"seed-iterator-for-testing-is-not-available","errorCode":null,"errorMessage":"Seed iterator for testing is not available.","messagePattern":"Seed iterator for testing is not available\\.","errorType":"exception","errorClass":"SeedIteratorNotAvailable","httpStatus":null,"severity":"error","filePath":"qlib/rl/trainer/vessel.py","lineNumber":64,"sourceCode":"    reward: Reward\n    trainer: Trainer\n\n    def assign_trainer(self, trainer: Trainer) -> None:\n        self.trainer = weakref.proxy(trainer)  # type: ignore\n\n    def train_seed_iterator(self) -> ContextManager[Iterable[InitialStateType]] | Iterable[InitialStateType]:\n        \"\"\"Override this to create a seed iterator for training.\n        If the iterable is a context manager, the whole training will be invoked in the with-block,\n        and the iterator will be automatically closed after the training is done.\"\"\"\n        raise SeedIteratorNotAvailable(\"Seed iterator for training is not available.\")\n\n    def val_seed_iterator(self) -> ContextManager[Iterable[InitialStateType]] | Iterable[InitialStateType]:\n        \"\"\"Override this to create a seed iterator for validation.\"\"\"\n        raise SeedIteratorNotAvailable(\"Seed iterator for validation is not available.\")\n\n    def test_seed_iterator(self) -> ContextManager[Iterable[InitialStateType]] | Iterable[InitialStateType]:\n        \"\"\"Override this to create a seed iterator for testing.\"\"\"\n        raise SeedIteratorNotAvailable(\"Seed iterator for testing is not available.\")\n\n    def train(self, vector_env: BaseVectorEnv) -> Dict[str, Any]:\n        \"\"\"Implement this to train one iteration. In RL, one iteration usually refers to one collect.\"\"\"\n        raise NotImplementedError()\n\n    def validate(self, vector_env: FiniteVectorEnv) -> Dict[str, Any]:\n        \"\"\"Implement this to validate the policy once.\"\"\"\n        raise NotImplementedError()\n\n    def test(self, vector_env: FiniteVectorEnv) -> Dict[str, Any]:\n        \"\"\"Implement this to evaluate the policy on test environment once.\"\"\"\n        raise NotImplementedError()\n\n    def log(self, name: str, value: Any) -> None:\n        # FIXME: this is a workaround to make the log at least show somewhere.\n        # Need a refactor in logger to formalize this.\n        if isinstance(value, (np.ndarray, list)):\n            value = np.mean(value)","sourceCodeStart":46,"sourceCodeEnd":82,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/rl/trainer/vessel.py#L46-L82","documentation":"SeedIteratorNotAvailable from the default `Vessel.test_seed_iterator` (qlib/rl/trainer/vessel.py:64). The trainer obtains test-environment initial states from this hook; the base class intentionally raises because it cannot know your task's test set.","triggerScenarios":"Calling `trainer.test(vessel)` (or a workflow step that evaluates on a test split) with a vessel that does not override `test_seed_iterator`.","commonSituations":"Running the standard train->test pipeline where only train/val hooks were implemented; renaming to `test_seed_iter` in a refactor; expecting the trainer to fall back to the validation iterator automatically (it does not).","solutions":["Override `test_seed_iterator` to yield the held-out test initial states.","Mirror the structure of `train_seed_iterator` with your test split (orders/dates excluded from train and val).","If test evaluation is optional in your script, gate it behind a config flag so the hook is only required when testing is enabled."],"exampleFix":"// before\ntrainer.test(my_vessel)  # vessel lacks test_seed_iterator\n// after\nclass MyVessel(Vessel):\n    def test_seed_iterator(self): return iter(test_orders)\ntrainer.test(my_vessel)","handlingStrategy":"validation","validationCode":"from qlib.rl.trainer.vessel import Vessel\n\ndef test_seeds_available(vessel: Vessel) -> bool:\n    return type(vessel).test_seed_iterator is not Vessel.test_seed_iterator","typeGuard":"def vessel_supports_testing(v) -> bool:\n    from qlib.rl.trainer.vessel import Vessel\n    return type(v).test_seed_iterator is not Vessel.test_seed_iterator","tryCatchPattern":"from qlib.rl.trainer.vessel import SeedIteratorNotAvailable\ntry:\n    metrics = trainer.test(vessel)\nexcept SeedIteratorNotAvailable:\n    log.warning(\"test seed iterator missing; skip evaluation\")\n    metrics = {}","preventionTips":["Gate trainer.test behind a config flag tied to the presence of a test seed hook.","Keep test seeds disjoint from train/val to avoid leaks and lifecycle confusion.","Implement all three seed hooks together when creating a vessel."],"tags":["rl","trainer","vessel","testing","not-implemented"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}