nautechsystems/nautilus_trader · error

canonical backtest result bytes do not use the canonical…

Error message

canonical backtest result bytes do not use the canonical encoding

What it means

After semantic canonicalization checks pass, `from_slice` re-serializes the normalized document and requires the resulting bytes to be byte-identical to the input. This is the strictest check: any difference in whitespace, key order, or number/text encoding from the canonical encoder fails with this message. It guarantees hashes/deduplication of result bytes are stable.

Solutions

  1. Use the exact bytes emitted by the canonical writer (do not pretty-print, reformat, or re-key the JSON)
  2. If you only have a reformatted copy, parse it, canonicalize via the library path (from_state or the canonical writer), and write canonical bytes first
  3. Compare `serde_json::to_vec(&normalized)` with your bytes to locate the encoding difference
  4. Validate with `jq -c .` that round-tripping to compact form matches — if it still differs, the issue is number/string encoding

Example fix

// before
let pretty = serde_json::to_vec_pretty(&result)?;
BacktestResult::from_slice(&pretty)?;
// after
let canonical = serde_json::to_vec(&result)?; // compact, canonical ordering
BacktestResult::from_slice(&canonical)?;
Defensive patterns

Strategy: validation

Validate before calling

import json
def roundtrip_equal(path):
    raw = open(path, 'rb').read()
    return json.dumps(json.loads(raw), separators=(',', ':'), ensure_ascii=False).encode() == raw

Try / catch

match BacktestResult::from_slice(&bytes) {
    Err(e) if e.to_string().contains("bytes do not use the canonical encoding") => {
        eprintln!("re-encode canonically before loading");
    }
    Err(e) => return Err(e),
    Ok(r) => r,
}

Prevention

When it happens

Trigger: Calling `from_slice` with pretty-printed JSON, JSON with different key ordering, alternate whitespace, differently formatted numbers, or escaped-vs-unescaped unicode — anything not produced by the library's canonical `serde_json::to_vec` of the normalized document.

Common situations: Pretty-printing a result file for readability then trying to load it back; reserializing with `jq` or Python's json (different spacing/key order); concatenating or trimming bytes.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors


AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/474e2c710681e9ab. Report an issue: GitHub.

Appendix: source

Thrown at crates/backtest/src/result.rs:198

    ///
    /// Inner records retain their NautilusTrader model serialization and are compared as semantic
    /// content. Producers must use the version 1 writer rather than construct records independently.
    ///
    /// # Errors
    ///
    /// Returns an error if the bytes are not a canonical version 1 result.
    pub fn from_slice(bytes: &[u8]) -> anyhow::Result<Self> {
        let document: Value = serde_json::from_slice(bytes)
            .map_err(|e| anyhow::anyhow!("invalid canonical backtest result JSON: {e}"))?;
        validate_document(&document)?;
        let mut normalized = document.clone();
        canonicalize_document(&mut normalized)?;
        anyhow::ensure!(
            normalized == document,
            "canonical backtest result violates the version 1 encoding rules"
        );
        let canonical = serde_json::to_vec(&normalized)?;
        anyhow::ensure!(
            canonical == bytes,
            "canonical backtest result bytes do not use the canonical encoding"
        );
        Ok(Self { document })
    }

    /// Returns the canonical compact UTF-8 JSON bytes without trailing data.
    ///
    /// # Errors
    ///
    /// Returns an error if the in-memory document cannot be serialized.
    pub fn to_bytes(&self) -> anyhow::Result<Vec<u8>> {
        Ok(serde_json::to_vec(&self.document)?)
    }

    /// Returns `blake3:` followed by the 32-byte BLAKE3 digest as 64 lowercase hex digits.
    ///
    /// # Errors

View on GitHub (pinned to 18893faf8b)