nautechsystems/nautilus_trader · error

second submit fills the slot

Error message

second submit fills the slot

What it means

Test panic from `.expect("second submit fills the slot")` at crates/event_store/src/writer/mod.rs:1410. After the writer thread is confirmed blocked at the gated append (appends_seen == 1), the second submit must enqueue into the now-free capacity-1 slot and return `Ok`; the expect panics if it is rejected. A rejection means the slot was still occupied, a halt already latched, or the retry loop timed out past the threshold.

Source

Thrown at crates/event_store/src/writer/mod.rs:1410

            EventStoreWriter::spawn(Box::new(backend), clock, halt, config).expect("spawn"),
        );

        writer.submit(entry_draft(10)).expect("first submit fits");

        let mut waited = Duration::ZERO;
        while appends_seen.load(Ordering::SeqCst) == 0 && waited < Duration::from_secs(2) {
            std::thread::sleep(Duration::from_millis(5));
            waited += Duration::from_millis(5);
        }
        assert_eq!(
            appends_seen.load(Ordering::SeqCst),
            1,
            "writer thread did not reach the gated append",
        );

        writer
            .submit(entry_draft(11))
            .expect("second submit fills the slot");

        // This submit stalls past the threshold and latches the halt
        let stalled = writer.submit(entry_draft(12)).expect_err("must stall");
        assert!(
            matches!(stalled, SubmitError::HaltSignaled { .. }),
            "was {stalled:?}",
        );

        // A second submitter now waits in the retry loop while the channel stays full
        let writer_for_thread = Arc::clone(&writer);
        let retrying = std::thread::spawn(move || writer_for_thread.submit(entry_draft(13)));
        std::thread::sleep(Duration::from_millis(20));

        // Release the gate: the freed slot must not rescue the retrying submit
        let (lock, cvar) = &*gate;
        *lock.lock() = true;
        cvar.notify_all();

View on GitHub (pinned to 18893faf8b)

Solutions

1. Increase `halt_threshold` so the second submit's brief retry window cannot trip the stall latch.
2. Confirm the writer thread really consumed the first entry (the `appends_seen == 1` assert) before the second submit; keep that poll bounded and generous.
3. Match on `SubmitError` to log whether the failure is `HaltSignaled` (threshold too tight) or `Closed` (thread halted) instead of a bare expect.
4. Ensure the gate starts closed (`Mutex::new(false)`) so the writer stays blocked and the slot semantics hold as the test assumes.

Example fix

// before
writer
    .submit(entry_draft(11))
    .expect("second submit fills the slot");
// after
writer
    .submit(entry_draft(11))
    .expect("second submit must fill the drained capacity-1 slot before any halt");

// Confirm the writer thread reached the gated append before filling the slot
assert_eq!(appends_seen.load(Ordering::SeqCst), 1,
    "writer thread did not reach the gated append");

match writer.submit(entry_draft(11)) {
    Ok(_) => {},
    Err(SubmitError::HaltSignaled { .. }) => eprintln!("halt_threshold expired before submit"),
    Err(SubmitError::Closed) => eprintln!("writer halted"),
}

writer.submit(entry_draft(11))
    .unwrap_or_else(|e| panic!("second submit rejected: {e:?}"));

When it happens

Trigger: Calling `submit` when the capacity-1 channel is not actually drained (writer thread never dequeued the first entry), or when the 50ms `halt_threshold` expired during polling so `HaltSignaled`/`Closed` is returned.

Common situations: Slow CI where the 5ms polling loop takes too long and the halt fires early; tests asserting on `appends_seen` without confirming the gate is held; fixture configs with too-small `halt_threshold` relative to polling overhead.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• heartbeat health margin should be shorter than the safety ti
• first submit fits
• second submit must be buffered by the clamped capacity
• submit accepted
• in-flight mutex poisoned

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/12722ab0fb19d4c7. Report an issue: GitHub.