{"record":{"id":"1fefa9de69dd083f","repo":"astrid-runtime/astrid","slug":"retained-bytes-exceed-logical-bytes","errorCode":null,"errorMessage":"retained bytes exceed logical bytes","messagePattern":"retained bytes exceed logical bytes","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/astrid-storage-chunker-evidence/src/metrics.rs","lineNumber":183,"sourceCode":"    let length = u64::try_from(bytes.len())?;\n    let identity = *blake3::hash(bytes).as_bytes();\n    match identities.entry(identity) {\n        std::collections::hash_map::Entry::Vacant(entry) => {\n            entry.insert(length);\n            *unique_bytes = checked_add(*unique_bytes, length, \"unique byte count\")?;\n        },\n        std::collections::hash_map::Entry::Occupied(entry) => {\n            if *entry.get() != length {\n                bail!(\"BLAKE3 evidence collision with inconsistent lengths\");\n            }\n        },\n    }\n    Ok(())\n}\n\nfn deduplication(logical_bytes: u64, retained_bytes: u64) -> Result<Deduplication> {\n    if retained_bytes > logical_bytes {\n        bail!(\"retained bytes exceed logical bytes\");\n    }\n    let saved_bytes = logical_bytes\n        .checked_sub(retained_bytes)\n        .ok_or_else(|| anyhow::anyhow!(\"retained bytes exceed logical bytes\"))?;\n    Ok(Deduplication {\n        retained_bytes,\n        saved_bytes,\n        retained_basis_points: basis_points(retained_bytes, logical_bytes)?,\n        saved_basis_points: basis_points(saved_bytes, logical_bytes)?,\n    })\n}\n\n/// Returns the ratio in basis points, rounded down.\n///\n/// Retained and saved ratios are deliberately calculated independently with\n/// this same rule. Deriving one as the complement of the other would round one\n/// side up whenever the exact ratio is fractional.\nfn basis_points(part: u64, total: u64) -> Result<u64> {","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/astrid-runtime/astrid/blob/affd8760f44190dbdfbec23403f4c4b642c33112/crates/astrid-storage-chunker-evidence/src/metrics.rs#L165-L201","documentation":"deduplication computes saved bytes as logical_bytes minus retained_bytes; it bails if retained bytes exceed logical bytes, which is mathematically impossible for a correct corpus measurement. This is a sanity invariant protecting the Deduplication result.","triggerScenarios":"Calling the Measurements summary with retained_bytes > logical_bytes — e.g. summing retained representations across overlapping records double-counted, or logical byte totals undercounted because some files were skipped.","commonSituations":"Measuring a subset of the corpus but summing retained bytes over the full set; mixing metrics from multiple runs; counting compressed/replicated bytes as both retained and additional logical bytes.","solutions":["Audit how logical_bytes and retained_bytes are accumulated so every retained byte is counted exactly once within the same logical corpus.","Recompute metrics from the complete corpus in a single pass.","Check that no representation records were added from a different/overlapping corpus."],"exampleFix":"// before\n// logical bytes from 3 of 4 files, retained bytes from all 4\n// after\n// visit all inputs in one pass so logical and retained totals cover the same corpus","handlingStrategy":"validation","validationCode":"if retained_bytes > logical_bytes { return Err(anyhow!(\"retained {} > logical {}\", retained_bytes, logical_bytes)); }","typeGuard":null,"tryCatchPattern":"match summary() {\n    Err(e) if e.to_string().contains(\"retained bytes exceed\") => recompute_from_full_corpus(),\n    other => other,\n}","preventionTips":["Count logical and retained bytes in the same single pass over the corpus","Never sum metrics from overlapping corpora","Ensure every retained representation is also counted in logical bytes exactly once"],"tags":["metrics","deduplication","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"affd8760f44190dbdfbec23403f4c4b642c33112","analyzedAt":"2026-09-09T21:28:12.402Z","contentChangedAt":"2026-09-09T21:28:12.402Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}