nautechsystems/nautilus_trader · error

Failed to parse index hash '{key}': {e}

Error message

Failed to parse index hash '{key}': {e}

What it means

read_index_hash reads the stored index hash bytes for a trader and deserializes them as JSON (HashMap). If the bytes are not valid JSON or do not match the expected map shape, the serde error is wrapped as 'Failed to parse index hash'. This means the internal key-index structure in Redis is corrupt or in an unexpected format.

Source

Thrown at crates/infrastructure/src/redis/queries.rs:722

        let index = Self::read_index_hash(con, trader_key, INDEX_ORDER_CLIENT).await?;
        Ok(index
            .into_iter()
            .map(|(k, v)| (ClientOrderId::from(k.as_str()), ClientId::from(v.as_str())))
            .collect())
    }

    async fn read_index_hash(
        con: &ConnectionManager,
        trader_key: &str,
        key: &str,
    ) -> anyhow::Result<HashMap<String, String>> {
        let result = Self::read(con, trader_key, key).await?;
        if result.is_empty() {
            return Ok(HashMap::new());
        }

        serde_json::from_slice(&result[0])
            .map_err(|e| anyhow::anyhow!("Failed to parse index hash '{key}': {e}"))
    }

    /// Loads all custom data for `trader_key` matching the given `data_type`.
    ///
    /// Keys are stored as `custom:<ts_init_020>:<uuid>`; value is full `CustomData` JSON.
    /// Scans all custom keys, deserializes, filters by `type_name` (full or short), metadata,
    /// and identifier to match SQL semantics, then sorts by `ts_init` ascending.
    ///
    /// # Errors
    ///
    /// Returns an error if scanning, bulk read, or deserialization fails.
    pub async fn load_custom_data(
        con: &ConnectionManager,
        trader_key: &str,
        data_type: &DataType,
    ) -> anyhow::Result<Vec<CustomData>> {
        let pattern = format!("{trader_key}{REDIS_DELIMITER}{CUSTOM}*");
        log::debug!("Loading custom data {pattern}");

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Dump the index key with redis-cli GET and confirm whether it is valid JSON.
  2. Match the SerializationEncoding to the one used when the index was written, or re-snapshot.
  3. Delete the corrupt index key(s) and rebuild them from the value keys.
  4. Upgrade/downgrade to the same version used to write the cache.

Example fix

// before: reading a MsgPack-written index with Json encoding
let idx = read_index_hash(con, trader_key, key).await?; // encoding: Json
// after: use the encoding the index was written with
let idx = read_index_hash(con, trader_key, key).await?; // encoding: MsgPack
Defensive patterns

Strategy: try-catch

Validate before calling

// Rust: validate index bytes are JSON before reading
fn index_bytes_valid(bytes: &[u8]) -> bool {
    serde_json::from_slice::<serde_json::Value>(bytes).is_ok()
}

Try / catch

match read_index_hash(&mut con, trader_key, key).await {
    Ok(index) => index,
    Err(e) if e.to_string().contains("Failed to parse index hash") => {
        // rebuild the index from value keys, then retry
        rebuild_index(&mut con, trader_key).await?;
        read_index_hash(&mut con, trader_key, key).await?
    }
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: Calling any loader that reads the index hash (orders/positions/custom data lookups via read_index_hash) when the index key holds non-JSON bytes, MsgPack data written with a different encoding, or a truncated write.

Common situations: Encoding config changed between write and read; index written by an older NautilusTrader version with a different index format; manual edits or flush/partial restores corrupting index keys.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/8a6337095858db97. Report an issue: GitHub.