tursodatabase/turso · error · anyhow::Error

MVCC logical log frame checksum mismatch at offset {offset}

Error message

MVCC logical log frame checksum mismatch at offset {offset}

What it means

Frame CRCs are chained: the chain seeds from crc32c of the header salt, and each frame's trailer stores crc32c_append(running_crc, header+payload+extension). A mismatch at the reported offset means this frame's bytes changed — or an earlier frame was inserted, dropped, or came from a log with a different salt, breaking the chain at this point. Even later frames that look intact cannot be trusted because the running CRC state is wrong.

Source

Thrown at cli/sync_server.rs:1090

        extension_size
    } else {
        0
    };
    let trailer_start = offset
        .checked_add(header_size)
        .and_then(|value| value.checked_add(payload_size))
        .and_then(|value| value.checked_add(extension_size))
        .ok_or_else(|| anyhow!("MVCC logical log frame offset overflow"))?;
    let frame_end = trailer_start
        .checked_add(MVCC_TX_TRAILER_SIZE)
        .ok_or_else(|| anyhow!("MVCC logical log frame end overflow"))?;
    if frame_end > log.len() {
        return Ok(None);
    }
    let expected_crc = crc32c::crc32c_append(running_crc, &log[offset..trailer_start]);
    let stored_crc = read_u32_le(log, trailer_start)?;
    if stored_crc != expected_crc {
        return Err(anyhow!(
            "MVCC logical log frame checksum mismatch at offset {offset}"
        ));
    }
    let end_magic = read_u32_le(log, trailer_start + 4)?;
    if end_magic != MVCC_TX_END_MAGIC {
        return Err(anyhow!(
            "invalid MVCC logical log frame end magic at offset {offset}"
        ));
    }
    Ok(Some((frame_end, stored_crc)))
}

fn read_u32_le(buf: &[u8], offset: usize) -> Result<u32> {
    let bytes = buf
        .get(offset..offset + 4)
        .ok_or_else(|| anyhow!("buffer too short for u32 at offset {offset}"))?;
    Ok(u32::from_le_bytes(bytes.try_into().unwrap()))
}

View on GitHub (pinned to bad083fafb)

Solutions

Re-pull the entire logical log from a known-good snapshot — one mismatch invalidates the chain and every later delta.
Never splice frames between logs or generations; the chain is salted per log header.
Compare the log with the producer's copy (sha256) to locate the first divergent byte and confirm the corruption is local.
If mismatches recur on one node, investigate storage durability (fsync behavior, failing hardware).

Example fix

// before
let snapshot = scan_mvcc_log(&log)?; // errors: frame checksum mismatch at offset N

// after: do not serve deltas past the break — re-pull the whole log
let snapshot = match scan_mvcc_log(&log) {
    Ok(snapshot) => snapshot,
    Err(err) if err.to_string().contains("frame checksum mismatch") => {
        rebootstrap_log_from_server()?
    }
    Err(err) => return Err(err),
};

Defensive patterns

Strategy: try-catch

Try / catch

match scan_mvcc_log(&log) {
    Ok(snapshot) => { /* serve deltas from crc_by_offset */ }
    Err(err) if err.to_string().contains("frame checksum mismatch") => {
        // chain broken: re-pull the whole log; deltas past the break are
        // untrustworthy even if later frames look intact
    }
    Err(err) => return Err(err),
}

Prevention

Re-pull the entire log on any frame CRC failure; never resume mid-chain.
Never splice frames between logs — the chain is salted by the header salt.
Ensure frames and trailers are written and fsynced together before announcement.
Hash-compare with the producer's copy to confirm where divergence began.

When it happens

Trigger: scan_mvcc_log walking frames where crc32c_append(running_crc, frame bytes) differs from the stored trailer CRC: bit corruption inside one frame, an earlier frame inserted or removed (the chain breaks at the next frame), or frames replayed against a log with a different header salt.

Common situations: Unreliable storage (failing disk, network filesystem); copying a log while it is being appended so frames are half-written; splicing frames between logs of different generations/salts; crash between frame write and trailer write.

Related errors

AI-assisted analysis of tursodatabase/turso@bad083fafb (2026-08-16). Data as JSON: /api/errors/e003f672eece9051. Report an issue: GitHub.