What causes "Instrument close key ID {instrument_id} did not match payload ID {}" in nautechsystems/nautilus_trader?

Inspect the named key and its payload's instrument_id; delete or move the misplaced payload.

Instrument close key ID {instrument_id} did not match payloa

Error message

Instrument close key ID {instrument_id} did not match payload ID {}

What it means

After deserializing an InstrumentClose, load_instrument_closes verifies that the instrument_id embedded in the payload matches the instrument_id parsed from the Redis key. A mismatch means the value stored under one instrument's key actually belongs to a different instrument — an internal consistency violation.

Source

Thrown at crates/infrastructure/src/redis/queries.rs:431

        con: &ConnectionManager,
        trader_key: &str,
        encoding: SerializationEncoding,
    ) -> anyhow::Result<AHashMap<InstrumentId, InstrumentClose>> {
        let prefix = format!("{trader_key}{REDIS_DELIMITER}{INSTRUMENT_CLOSES}{REDIS_DELIMITER}");
        let pattern = format!("{prefix}*");
        log::debug!("Loading {pattern}");

        let mut con = con.clone();
        let keys = Self::scan_keys(&mut con, pattern).await?;
        let values = Self::read_bulk(&con, &keys).await?;
        let mut closes = AHashMap::with_capacity(keys.len());

        for (key, value) in keys.into_iter().zip(values) {
            let instrument_id = parse_instrument_key(&key, &prefix)?;
            let value = value
                .ok_or_else(|| anyhow::anyhow!("Instrument close not found in Redis: {key}"))?;
            let close: InstrumentClose = Self::deserialize_payload(encoding, &value)?;
            anyhow::ensure!(
                close.instrument_id == instrument_id,
                "Instrument close key ID {instrument_id} did not match payload ID {}",
                close.instrument_id,
            );
            closes.insert(instrument_id, close);
        }

        log::debug!("Loaded {} instrument close(s)", closes.len());
        Ok(closes)
    }

    /// Loads all synthetic instruments for `trader_key` using the specified `encoding`.
    ///
    /// # Errors
    ///
    /// Returns an error if scanning keys or reading synthetic instrument data fails.
    pub async fn load_synthetics(
        con: &ConnectionManager,

View on GitHub (pinned to 18893faf8b)

Solutions

Inspect the named key and its payload's instrument_id; delete or move the misplaced payload.
Re-snapshot the cache so keys and payload IDs are regenerated consistently.
Check for concurrent writers using the same trader_id/database writing conflicting data.
If instrument IDs were renamed, run a migration script to rewrite payloads' instrument_id fields to match new keys.

Example fix

// before: key says EURUSD.IDEAL but payload was written for EURUSD.SIM -> mismatch
// after: rewrite payload under correct key / fix payload ID
con.set("closes:EURUSD.SIM", serialize(InstrumentClose { instrument_id: "EURUSD.SIM", .. }));

Defensive patterns

Strategy: validation

Validate before calling

// Rust: check key/payload ID agreement before insert
fn ids_agree(key_id: &InstrumentId, close: &InstrumentClose) -> bool {
    close.instrument_id == *key_id
}

Try / catch

match load_instrument_closes(&mut con, trader_key, encoding).await {
    Ok(closes) => closes,
    Err(e) if e.to_string().contains("did not match payload ID") => {
        tracing::error!("corrupt instrument-close entry: {e}; re-snapshot required");
        return Err(e);
    }
    Err(e) => return Err(e),
}

Prevention

Use a dedicated trader_id and Redis DB per instance to avoid concurrent writers clobbering keys.
Never rename/copy value keys manually; rewrite payloads through the library so embedded IDs stay in sync.
Re-snapshot the cache after instrument symbol/venue migrations.

When it happens

Trigger: Calling load_instrument_closes / load_all when a value key was overwritten by a payload for a different instrument, keys/values were mismatched in the index, or an instrument_id was renamed (e.g. venue/symbol change) while old payloads persisted.

Common situations: Manually copying/renaming Redis keys; symbol migrations or instrument re-definitions between versions; concurrent writers clobbering each other's value keys; a bug in a custom snapshot tool.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/5c0f130a0dab8bdf. Report an issue: GitHub.