{"record":{"id":"474e2c710681e9ab","repo":"nautechsystems/nautilus_trader","slug":"canonical-backtest-result-bytes-do-not-use-the-can","errorCode":null,"errorMessage":"canonical backtest result bytes do not use the canonical encoding","messagePattern":"canonical backtest result bytes do not use the canonical encoding","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/backtest/src/result.rs","lineNumber":198,"sourceCode":"    ///\n    /// Inner records retain their NautilusTrader model serialization and are compared as semantic\n    /// content. Producers must use the version 1 writer rather than construct records independently.\n    ///\n    /// # Errors\n    ///\n    /// Returns an error if the bytes are not a canonical version 1 result.\n    pub fn from_slice(bytes: &[u8]) -> anyhow::Result<Self> {\n        let document: Value = serde_json::from_slice(bytes)\n            .map_err(|e| anyhow::anyhow!(\"invalid canonical backtest result JSON: {e}\"))?;\n        validate_document(&document)?;\n        let mut normalized = document.clone();\n        canonicalize_document(&mut normalized)?;\n        anyhow::ensure!(\n            normalized == document,\n            \"canonical backtest result violates the version 1 encoding rules\"\n        );\n        let canonical = serde_json::to_vec(&normalized)?;\n        anyhow::ensure!(\n            canonical == bytes,\n            \"canonical backtest result bytes do not use the canonical encoding\"\n        );\n        Ok(Self { document })\n    }\n\n    /// Returns the canonical compact UTF-8 JSON bytes without trailing data.\n    ///\n    /// # Errors\n    ///\n    /// Returns an error if the in-memory document cannot be serialized.\n    pub fn to_bytes(&self) -> anyhow::Result<Vec<u8>> {\n        Ok(serde_json::to_vec(&self.document)?)\n    }\n\n    /// Returns `blake3:` followed by the 32-byte BLAKE3 digest as 64 lowercase hex digits.\n    ///\n    /// # Errors","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/backtest/src/result.rs#L180-L216","documentation":"After semantic canonicalization checks pass, `from_slice` re-serializes the normalized document and requires the resulting bytes to be byte-identical to the input. This is the strictest check: any difference in whitespace, key order, or number/text encoding from the canonical encoder fails with this message. It guarantees hashes/deduplication of result bytes are stable.","triggerScenarios":"Calling `from_slice` with pretty-printed JSON, JSON with different key ordering, alternate whitespace, differently formatted numbers, or escaped-vs-unescaped unicode — anything not produced by the library's canonical `serde_json::to_vec` of the normalized document.","commonSituations":"Pretty-printing a result file for readability then trying to load it back; reserializing with `jq` or Python's json (different spacing/key order); concatenating or trimming bytes.","solutions":["Use the exact bytes emitted by the canonical writer (do not pretty-print, reformat, or re-key the JSON)","If you only have a reformatted copy, parse it, canonicalize via the library path (from_state or the canonical writer), and write canonical bytes first","Compare `serde_json::to_vec(&normalized)` with your bytes to locate the encoding difference","Validate with `jq -c .` that round-tripping to compact form matches — if it still differs, the issue is number/string encoding"],"exampleFix":"// before\nlet pretty = serde_json::to_vec_pretty(&result)?;\nBacktestResult::from_slice(&pretty)?;\n// after\nlet canonical = serde_json::to_vec(&result)?; // compact, canonical ordering\nBacktestResult::from_slice(&canonical)?;","handlingStrategy":"validation","validationCode":"import json\ndef roundtrip_equal(path):\n    raw = open(path, 'rb').read()\n    return json.dumps(json.loads(raw), separators=(',', ':'), ensure_ascii=False).encode() == raw","typeGuard":null,"tryCatchPattern":"match BacktestResult::from_slice(&bytes) {\n    Err(e) if e.to_string().contains(\"bytes do not use the canonical encoding\") => {\n        eprintln!(\"re-encode canonically before loading\");\n    }\n    Err(e) => return Err(e),\n    Ok(r) => r,\n}","preventionTips":["Disable pretty-printing for files intended for from_slice","Don't reformat saved results with jq/editors; keep the original bytes","Store a hash of canonical bytes to detect accidental reformatting"],"tags":["json","canonicalization","bytes","backtest"],"backgroundTag":"checksum-mismatch","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}