microsoft/qlib · error · RuntimeError

State queue is already exhausted, but the environment is sti

Error message

State queue is already exhausted, but the environment is still receiving action.

What it means

RuntimeError in `EnvWrapper.step` (qlib/rl/utils/env_wrapper.py:202). `step` requires an active simulator seeded by a prior successful reset; if `seed_iterator` is None (exhausted, wrapper dead) the wrapper can no longer accept actions and raises. A dead wrapper should have been recycled after the NaN observation returned by the final reset.

Source

Thrown at qlib/rl/utils/env_wrapper.py:202

            obs = self.state_interpreter(sim_state)

            self.status["obs_history"].append(obs)

            return obs

        except StopIteration:
            # The environment should be recycled because it's in a dead state.
            self.seed_iterator = None
            return generate_nan_observation(self.observation_space)

    def step(self, policy_action: PolicyActType, **kwargs: Any) -> Tuple[ObsType, float, bool, InfoDict]:
        """Environment step.

        See the code along with comments to get a sequence of things happening here.
        """

        if self.seed_iterator is None:
            raise RuntimeError("State queue is already exhausted, but the environment is still receiving action.")

        # Clear the logged information from last step
        self.logger.reset()

        # Action is what we have got from policy
        self.status["action_history"].append(policy_action)
        action = self.action_interpreter(self.simulator.get_state(), policy_action)

        # This update must be after action interpreter and before simulator.
        self.status["cur_step"] += 1

        # Use the converted action of update the simulator
        self.simulator.step(action)

        # Update "done" first, as this status might be used by reward_fn later
        done = self.simulator.done()
        self.status["done"] = done

View on GitHub (pinned to 79633dd950)

Solutions

1. After each reset, check the observation validity (e.g. `finite_env.is_invalid(obs)`) and stop stepping that environment when invalid.
2. In vectorized settings rely on qlib's vector env to mask/recycle exhausted environments rather than stepping every slot uniformly.
3. Size the seed iterator to the intended number of episodes so exhaustion aligns with the end of the phase.

Example fix

// before
obs = env.reset()
while True:
    obs, rew, done, info = env.step(policy(obs))  # keeps stepping even after NaN obs
// after
from qlib.rl.utils.finite_env import is_invalid
obs = env.reset()
while not is_invalid(obs):
    obs, rew, done, info = env.step(policy(obs))

from qlib.rl.utils.finite_env import is_invalid

def should_step(wrapper, obs) -> bool:
    return wrapper.seed_iterator is not None and not is_invalid(obs)

def env_step_safe(wrapper) -> bool:
    return getattr(wrapper, "seed_iterator", None) is not None

try:
    obs, rew, done, info = wrapper.step(action)
except RuntimeError as e:
    if "still receiving action" in str(e):
        break  # environment exhausted: end rollout for this env
    raise

When it happens

Trigger: Ignoring the NaN terminal observation that reset returns when the seed queue is exhausted and continuing to call `step()`; a policy loop that doesn't check the done/invalid flag on the last environment in a FiniteVectorEnv; manual stepping after the finite episode set is spent.

Common situations: Custom rollout code that stops only on `done` but the wrapper dies at data exhaustion before done; aggregation code that steps all envs a fixed number of times regardless of validity; races where one worker's seed iterator finishes earlier than others'.

Related errors

• You can trying to get a state from a dead environment wrappe
• DataQueue can not activate twice.
• Need to call activate() to launch a daemon worker to produce
• Render is not implemented in EnvWrapper.
• model is not fitted yet!

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/c69695b5d6f59495. Report an issue: GitHub.