nautechsystems/nautilus_trader · error

Failed to deserialize msgpack `payload

Error message

Failed to deserialize msgpack `payload`: {e}

What it means

deserialize_payload decodes raw bytes back into T using rmp_serde::from_slice (MsgPack) or serde_json::from_slice (Json). This error wraps rmp_serde decode failures: the bytes are not valid msgpack or do not match the shape of T. SBE and Capnp are explicitly rejected before this branch.

Solutions

1. Verify the encoding parameter matches the encoding used at write time (check the entry's `encoding` header).
2. Ensure the target type T matches the struct that was serialized (same crate version on reader and writer).
3. Inspect the raw bytes with rmp_serde::from_slice::<rmpv::Value> to diagnose the mismatch.
4. Re-write the affected cache entries after aligning versions.

Example fix

// before: mismatched encoding
let v: MyType = deserialize_payload(SerializationEncoding::MsgPack, &bytes)?;
// after: use the encoding recorded when the payload was written
let v: MyType = deserialize_payload(encoding_from_header, &bytes)?;

// confirm bytes are msgpack before typed decode
let probe: Result<rmpv::Value, _> = rmp_serde::from_slice(bytes);
if probe.is_err() { return Err("payload is not valid msgpack".into()); }

match deserialize_payload::<MyType>(encoding, bytes) {
    Ok(v) => use(v),
    Err(e) if e.to_string().contains("Failed to deserialize") => {
        log::error!("payload shape/encoding mismatch: {e}");
        // fall back to default or re-fetch after version alignment
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling deserialize_payload::<T>(SerializationEncoding::MsgPack, bytes) where bytes were not produced by serialize_payload with MsgPack, were truncated/corrupted, or encode a different type's shape than T.

Common situations: Reading Redis cache entries written by a different app version whose payload struct changed fields; decoding a JSON-encoded payload with the MsgPack encoding parameter; corrupted/truncated bytes fetched from Redis.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• Failed to convert value to target type
• Failed to decode position replay state
• Failed to deserialize json `payload
• Failed to serialize msgpack `payload
• Invalid encoding format

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/947c227a0f365c07. Report an issue: GitHub.

Appendix: source

Thrown at crates/infrastructure/src/redis/queries.rs:115

            }
            SerializationEncoding::Capnp => {
                anyhow::bail!("Cap'n Proto encoding is not supported for Redis cache payloads")
            }
        }
    }

    /// Deserializes the given byte slice `payload` into type `T` using the specified `encoding`.
    ///
    /// # Errors
    ///
    /// Returns an error if deserialization from the chosen encoding fails or converting to the target type fails.
    pub fn deserialize_payload<T: DeserializeOwned>(
        encoding: SerializationEncoding,
        payload: &[u8],
    ) -> anyhow::Result<T> {
        let mut value = match encoding {
            SerializationEncoding::MsgPack => rmp_serde::from_slice(payload)
                .map_err(|e| anyhow::anyhow!("Failed to deserialize msgpack `payload`: {e}"))?,
            SerializationEncoding::Json => serde_json::from_slice(payload)
                .map_err(|e| anyhow::anyhow!("Failed to deserialize json `payload`: {e}"))?,
            SerializationEncoding::Sbe => {
                anyhow::bail!("SBE encoding is not supported for Redis cache payloads")
            }
            SerializationEncoding::Capnp => {
                anyhow::bail!("Cap'n Proto encoding is not supported for Redis cache payloads")
            }
        };

        convert_timestamp_strings(&mut value);

        serde_json::from_value(value)
            .map_err(|e| anyhow::anyhow!("Failed to convert value to target type: {e}"))
    }

    /// Scans Redis for keys matching the given `pattern`.
    ///

View on GitHub (pinned to 18893faf8b)