tracel-ai/burn · error

Can send message to autobatcher.

Error message

Can send message to autobatcher.

What it means

AsyncPolicy::increment_agents sends an IncrementAgents message to the autobatcher thread via a std::sync::mpsc channel and panics if send fails. send only fails when the receiving side of the channel has been closed, which happens once the autobatcher thread has exited — either because all senders were dropped (it saw a RecvError and broke out of its loop) or because it panicked on an earlier message. After this panic the AsyncPolicy handle is permanently unusable.

Solutions

1. Check logs for the original autobatcher-thread panic or 'Error in AsyncPolicy' message; fix the root cause that killed the thread first.
2. Recreate the AsyncPolicy (AsyncPolicy::new) once the server thread has died — a dead handle cannot be revived.
3. Avoid panics in the inner policy (validate observation shapes/batching) so the server thread never unwinds.
4. If a panic in the server thread is expected to be possible, wrap the thread body in catch_unwind or replace expects with error logging plus a restart mechanism.

Example fix

// before
let policy = policy.clone(); // stale clone from previous run; server thread already dead
policy.increment_agents(1);
// after
let policy = AsyncPolicy::new(autobatch_size, inner_policy.clone()); // rebuild after server death
policy.increment_agents(1);

// Before using a long-lived AsyncPolicy, verify the server is responsive
let probe = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
    let mut p = policy.clone();
    p.increment_agents(0);
}));
if probe.is_err() { rebuild_policy(); }

let ok = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| policy.increment_agents(1)));
if ok.is_err() {
    log::warn!("inference server dead; rebuilding AsyncPolicy");
    policy = AsyncPolicy::new(autobatch_size, inner_policy.clone());
}

When it happens

Trigger: Calling increment_agents on an AsyncPolicy after the inference thread has terminated: an earlier expect panic in the autobatcher loop (e.g. a failed action reply send or a panic in inner_policy.action/forward), or every clone of the AsyncPolicy being dropped and recreated so the old thread exited with RecvError while a stale clone is still used.

Common situations: Multi-threaded RL environments where one worker's panic poisons the shared inference server and remaining workers then panic on their next increment_agents/action call; holding an AsyncPolicy clone across a restart of the training loop; a previous operation panicked inside the inner policy (e.g. shape mismatch) killing the server thread.

Related errors

• AsyncPolicy should be able to receive policy state.
• Autobatcher should be able to send current policy state.
• Should be able to send message to inference_server
• AsyncPolicy should be able to send policy state.
• AsyncPolicy should receive queued actions.

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/697486dd2f12f2ac. Report an issue: GitHub.

Appendix: source

Thrown at crates/burn-rl/src/policy/async_policy.rs:241

                    },
                    Err(err) => {
                        log::error!("Error in AsyncPolicy : {}", err);
                        break;
                    }
                }
            }
        });

        Self {
            inference_state_sender: sender,
        }
    }

    /// Increment the number of agents using the inference server.
    pub fn increment_agents(&self, num: usize) {
        self.inference_state_sender
            .send(InferenceMessage::IncrementAgents(num))
            .expect("Can send message to autobatcher.")
    }

    /// Decrement the number of agents using the inference server.
    pub fn decrement_agents(&self, num: usize) {
        self.inference_state_sender
            .send(InferenceMessage::DecrementAgents(num))
            .expect("Can send message to autobatcher.")
    }
}

impl<P> Policy for AsyncPolicy<P>
where
    P: Policy + Send + 'static,
{
    type ActionContext = P::ActionContext;
    type PolicyState = P::PolicyState;

    type Observation = P::Observation;

View on GitHub (pinned to d16f7ba2ed)