clockworklabs/SpacetimeDB · critical

Write skew, you need to implement retries my man, T-dawg.

Error message

Write skew, you need to implement retries my man, T-dawg.

What it means

A `todo!()` panic in the subscription actor: the transaction that evaluates and broadcasts subscription updates failed to commit with `WriteConflict` (write skew against a concurrent transaction), and retrying that commit is not implemented yet. The informal message marks a known unimplemented path, so the server aborts instead of retrying.

Source

Thrown at crates/core/src/subscription/module_subscription_actor.rs:203

/// `Ok` side of a [`CommitAndBroadcastEventResult`].
pub struct CommitAndBroadcastEventSuccess {
    pub tx_offset: TransactionOffset,
    pub event: Arc<ModuleEvent>,
    pub metrics: ExecutionMetrics,
}

/// Commits `tx`
/// and evaluates and broadcasts subscriptions updates.
pub(crate) fn commit_and_broadcast_event(
    subs: &ModuleSubscriptions,
    client: Option<Arc<ClientConnectionSender>>,
    event: ModuleEvent,
    tx: MutTxId,
) -> CommitAndBroadcastEventSuccess {
    match subs.commit_and_broadcast_event(client, event, tx).unwrap() {
        Ok(res) => res,
        Err(WriteConflict) => todo!("Write skew, you need to implement retries my man, T-dawg."),
    }
}

type AssertTxFn = Arc<dyn Fn(&Tx) + Send + Sync + 'static>;
type SubscriptionUpdate =
    ws_v1::FormatSwitch<ws_v1::TableUpdate<ws_v1::BsatnFormat>, ws_v1::TableUpdate<ws_v1::JsonFormat>>;
type FullSubscriptionUpdate =
    ws_v1::FormatSwitch<ws_v1::DatabaseUpdate<ws_v1::BsatnFormat>, ws_v1::DatabaseUpdate<ws_v1::JsonFormat>>;

struct CompiledQueryBatch {
    queries: Vec<Arc<Plan>>,
    physical_plans: HashMap<QueryHash, Vec<ProjectPlan>>,
    auth: AuthCtx,
    mut_tx: MutTxId,
    compile_timer: HistogramTimer,
}

#[derive(Clone, Copy)]

View on GitHub (pinned to 9e0d92412f)

Solutions

1. Reduce contention: batch or stagger writes to rows covered by the same subscription
2. Retry the failed reducer call from the client with backoff — the server will not retry it for you
3. Upgrade SpacetimeDB: retry handling on this path is tracked as a todo and may exist in newer releases
4. Report the panic with a reproduction if it persists on the latest version

Example fix

// before
client.call_reducer("increment").await?; // server panics under write skew

// after
for attempt in 0..5u32 {
    match client.call_reducer("increment").await {
        Ok(_) => break,
        Err(e) if attempt < 4 => {
            tokio::time::sleep(std::time::Duration::from_millis(50 * (1 << attempt))).await;
        }
        Err(e) => return Err(e),
    }
}

for attempt in 0..5u32 {
    match client.call_reducer("increment").await {
        Ok(_) => break,
        Err(e) if attempt < 4 => {
            tokio::time::sleep(std::time::Duration::from_millis(50 * (1 << attempt))).await;
        }
        Err(e) => return Err(e),
    }
}

When it happens

Trigger: A workload where the subscription-evaluation transaction reads rows that another committed transaction concurrently modified, so `commit_and_broadcast_event` loses the race and returns Err(WriteConflict). For example, two reducers mutating rows covered by the same subscription at the same moment.

Common situations: High-concurrency writes to subscribed tables (leaderboards, counters, chat); load tests with many parallel reducers; scheduled or init reducers racing client reducers on the same tables.

Related errors

• a row was a sequence trigger but there was no generated colu
• Error getting jwt: {errno}
• {errno}
• length didn't fit in `u32`
• Unknown HTTP version: {:?}

AI-assisted analysis of clockworklabs/SpacetimeDB@9e0d92412f (2026-08-20). Data as JSON: /api/errors/74c3d7f9b4285b84. Report an issue: GitHub.