nautechsystems/nautilus_trader · error

Failed to convert value to target type: {e}

Error message

Failed to convert value to target type: {e}

What it means

After the raw JSON bytes parse successfully and timestamp strings are normalized, serde_json::from_value converts the intermediate serde_json::Value into the concrete type T. This error is thrown when the JSON structure is valid JSON but its shape/field types do not match T (missing fields, wrong types, unexpected nulls).

Source

Thrown at crates/infrastructure/src/redis/queries.rs:129

        payload: &[u8],
    ) -> anyhow::Result<T> {
        let mut value = match encoding {
            SerializationEncoding::MsgPack => rmp_serde::from_slice(payload)
                .map_err(|e| anyhow::anyhow!("Failed to deserialize msgpack `payload`: {e}"))?,
            SerializationEncoding::Json => serde_json::from_slice(payload)
                .map_err(|e| anyhow::anyhow!("Failed to deserialize json `payload`: {e}"))?,
            SerializationEncoding::Sbe => {
                anyhow::bail!("SBE encoding is not supported for Redis cache payloads")
            }
            SerializationEncoding::Capnp => {
                anyhow::bail!("Cap'n Proto encoding is not supported for Redis cache payloads")
            }
        };

        convert_timestamp_strings(&mut value);

        serde_json::from_value(value)
            .map_err(|e| anyhow::anyhow!("Failed to convert value to target type: {e}"))
    }

    /// Scans Redis for keys matching the given `pattern`.
    ///
    /// # Errors
    ///
    /// Returns an error if the Redis scan operation fails.
    pub async fn scan_keys(
        con: &mut ConnectionManager,
        pattern: String,
    ) -> anyhow::Result<Vec<String>> {
        let mut result = Vec::new();
        let mut cursor = 0u64;

        loop {
            let scan_result: (u64, Vec<String>) = redis::cmd("SCAN")
                .arg(cursor)
                .arg("MATCH")

View on GitHub (pinned to 18893faf8b)

Solutions

1. Read the serde error detail in {e}: it names the missing/invalid field path — fix the data at that path in Redis.
2. Align reader/writer versions or re-write the cache with the current schema version.
3. Migrate old payloads (e.g. script to re-serialize stored JSON into the new struct shape).
4. If a field type changed (string vs number), normalize the stored values.

Example fix

// before: old payload has ts_init as RFC3339 string, struct expects integer nanos
{"ts_init": "2024-01-01T00:00:00Z"}
// after: normalize to integer nanoseconds expected by the struct
{"ts_init": 1704067200000000000}

// Pre-check: payload fits target type
fn fits<T: serde::de::DeserializeOwned>(bytes: &[u8]) -> bool {
    serde_json::from_slice::<T>(bytes).is_ok()
}

fn is_complete_object(v: &serde_json::Value) -> bool {
    v.as_object().map(|o| !o.is_empty()).unwrap_or(false)
}

match deserialize_payload::<MyData>(encoding, &value) {
    Ok(data) => data,
    Err(e) if e.to_string().contains("convert value to target type") => {
        tracing::warn!("schema drift for stored payload: {e}");
        Default::default() // or migrate/skip the record
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling deserialize_payload with data whose JSON fields do not match struct T: e.g. a numeric field stored as string (not corrected by convert_timestamp_strings), a missing required field, or a changed field name between writer and reader versions.

Common situations: Schema drift between write and read versions of NautilusTrader; hand-edited Redis values; data written by a different component with slightly different field naming; an Optional field made required in a newer version while old data lacks it.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Failed to deserialize json `payload`: {e}
• Failed to decode position replay state: {e}
• fee exponent must be a decimal number or numeric string
• Invalid stream message format: {stream_msg:?}
• Invalid stream field key: {stream_msg:?}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/39dd9e66225d6624. Report an issue: GitHub.