nautechsystems/nautilus_trader · critical · anyhow::Error
OCM state lock poisoned
Error message
OCM state lock poisoned
What it means
ocm_state is a std::sync::Mutex<OcmState> shared by the OCM (order change message) reconciliation path; lock() returned a PoisonError, meaning some thread panicked while holding this mutex. From that point every OCM poll (fetch_fill_reports_via_http) fails with this message until the process restarts. The real defect is the earlier panic in the logs, not this error.
Solutions
- Search logs backwards from this error for the originating panic and its backtrace; fix or report that panic.
- Restart the trading node — a poisoned std Mutex never recovers in-process.
- If reproducible, capture the CurrentOrderSummary/stream update that triggered the panic and file an issue with the adapter maintainers.
Defensive patterns
Strategy: try-catch
Try / catch
match reports_result {
Ok(reports) => { /* process */ }
Err(e) if e.to_string().contains("lock poisoned") => {
log::error!("OCM state poisoned by earlier panic; restart required");
// halt trading safely: cancel open orders if possible, then exit
std::process::exit(1);
}
Err(e) => log::warn!("OCM poll failed: {e}"),
} Prevention
- Treat any panic in the Betfair adapter as critical: run the node with panic backtraces and log capture enabled.
- Monitor for the original panic — 'lock poisoned' is always a symptom, not the cause.
- Supervise the node process (systemd/k8s) so a poisoned state triggers a clean restart.
- Report the triggering payload to maintainers so the panic is fixed.
When it happens
Trigger: Any panic inside a critical section holding ocm_state (fill-report building, customer_order_ref registration in the submit paths) poisons the lock; a subsequent generate_fill_reports call then hits the poisoned lock at execution.rs:3038.
Common situations: An upstream panic (decimal parse, unwrap, index) during order reconciliation masks itself as 'lock poisoned' on all later OCM polls; common after an unexpected venue payload triggers a panic in the parsing path.
Related errors
- in-flight mutex poisoned
- slots lock poisoned
- wallet balance mutex poisoned
- 16 lowercase hex bytes are valid TradeId
- AccountId contains '-'
AI-assisted analysis of nautechsystems/nautilus_trader@d1527c24af (2026-08-27).
Data as JSON: /api/errors/1bfda39d8d7c2368.
Report an issue: GitHub.
Appendix: source
Thrown at crates/adapters/betfair/src/execution.rs:2040
self.emitter
.emit_order_denied(&order, &OrderDeniedReason::StreamReconciling.to_string());
return Ok(());
}
if order.is_closed() {
log::warn!("Cannot submit closed order {}", order.client_order_id());
return Ok(());
}
let instrument_id = order.instrument_id();
let market_id = extract_market_id(&instrument_id)?;
let (selection_id, handicap) = extract_selection_id(&instrument_id)?;
let instruction = create_place_instruction(&order, selection_id, handicap)?;
let collision = self
.ocm_state
.lock()
.map_err(|_| anyhow::anyhow!("OCM state lock poisoned"))?
.register_submission(order.client_order_id(), order.strategy_id())
.err();
if let Some(customer_order_ref) = collision {
let reason = customer_order_ref_collision_reason(&customer_order_ref);
log::warn!("Denying submit for {}: {reason}", order.client_order_id(),);
self.emitter.emit_order_denied(&order, &reason);
return Ok(());
}
let market_version = self.get_market_version(&instrument_id);
let params = PlaceOrdersParams {
market_id,
instructions: vec![instruction],
customer_ref: Some(order_customer_ref()),
market_version,
customer_strategy_ref: None,View on GitHub (pinned to d1527c24af)