clockworklabs/SpacetimeDB · critical

When replaying `st_column` update: `table_primary_key` shoul

Error message

When replaying `st_column` update: `table_primary_key` should be a single column, but found {col_list:?}

What it means

During replay of a schema-altering st_table update, reschema_table_for_st_table_update maps the stored table_primary_key list into the in-memory schema via ColList::as_singleton, which requires exactly one column. The replayed row carries a multi-column list, so the update cannot be applied and replay fails. The locking datastore models primary keys as a single column (identity/sequence), so a composite PK in st_table is un-replayable.

Source

Thrown at crates/datastore/src/locking_tx_datastore/replay.rs:852

                .expect("`st_table` row should conform to `st_table` schema")
        }

        let referenced_table_id = get_table_id(new_st_table_entry);
        self.iter_by_col_eq(ST_TABLE_ID, StTableFields::TableId, &referenced_table_id.into())
            .expect("`st_table` should exist")
            .any(|row_ref| row_ref.pointer() != new_st_table_entry.pointer())
    }

    /// Update the in-memory table structure for the table described by `row`,
    /// in response to replay of a schema-altering migration.
    fn reschema_table_for_st_table_update(&mut self, row: StTableRow) -> Result<()> {
        // We only need to update if we've already constructed the in-memory table structure.
        // If we haven't yet, then `self.get_table_and_blob_store_or_create` will see the correct schema
        // when it eventually runs.
        if let Ok((table, ..)) = self.get_table_and_blob_store_mut(row.table_id) {
            table.with_mut_schema(|schema| -> Result<()> {
                schema.table_access = row.table_access;
                schema.primary_key = row.table_primary_key.map(|col_list| col_list.as_singleton().ok_or_else(|| anyhow::anyhow!("When replaying `st_column` update: `table_primary_key` should be a single column, but found {col_list:?}"))).transpose()?;
                schema.table_name = row.table_name;
                if row.table_type == schema.table_type {
                    Ok(())
                } else {
                    Err(anyhow::anyhow!(
                    "When replaying `st_column` update: `table_type` should not have changed, but previous schema has {:?} and new schema has {:?}",
                    schema.table_type,
                    row.table_type,
                ).into())
}
            })?;
        }
        Ok(())
    }

    /// Mark all `st_column` rows which refer to the same column as `st_column_row`
    /// other than the one at `row_pointer` as outdated
    /// by storing them in [`Self::replay_columns_to_ignore`].

View on GitHub (pinned to 6dee26c6ef)

Solutions

  1. Replay with the same spacetimedb version that wrote the commitlog
  2. If composite primary keys are required, upgrade every node that will open the database
  3. Restore from a snapshot taken with a compatible version
  4. Report the incompatibility — the message includes the offending col list
Defensive patterns

Strategy: fallback

Try / catch

Wrap database open; on this schema-mismatch error abort the open, report the version pair in the message to operators, and fall back to a snapshot from a compatible version.

Prevention

When it happens

Trigger: A commitlog containing an st_table row whose table_primary_key list has more than one element — written by a different or experimental spacetimedb build, hand-crafted, or by a newer version introducing composite PKs that an older binary then tries to replay.

Common situations: Version skew between the binary that wrote the log and the one replaying it; replaying logs from experimental feature branches; downgrade compatibility testing.

Related errors


AI-assisted analysis of clockworklabs/SpacetimeDB@6dee26c6ef (2026-08-20). Data as JSON: /api/errors/ed187519c99937b9. Report an issue: GitHub.