snapshot file {} for cf {} checksum mismatches, real checksu

Error message

snapshot file {} for cf {} checksum mismatches, real checksum {}, expected checksum {}

What it means

Immediately after the size check, the same snapshot finalization loop in components/raftstore/src/store/snap.rs:1272 compares each CF file's finalized CRC (from write_digest) with the checksum recorded in the snapshot meta. On mismatch it returns InvalidData: "snapshot file {} for cf {} checksum mismatches, real checksum {}, expected checksum {}". The content matches the declared size but the bytes differ — silent corruption is being rejected.

Source

Thrown at components/raftstore/src/store/snap.rs:1272

                file_for_recving.file.sync_all()?;

                if file_for_recving.written_size != cf_file.size[i] {
                    return Err(io::Error::new(
                        ErrorKind::InvalidData,
                        format!(
                            "snapshot file {} for cf {} size mismatches, \
                            real size {}, expected size {}",
                            cf_file.path.display(),
                            cf_file.cf,
                            file_for_recving.written_size,
                            cf_file.size[i]
                        ),
                    ));
                }

                let checksum = file_for_recving.write_digest.finalize();
                if checksum != cf_file.checksum[i] {
                    return Err(io::Error::new(
                        ErrorKind::InvalidData,
                        format!(
                            "snapshot file {} for cf {} checksum \
                            mismatches, real checksum {}, expected \
                            checksum {}",
                            cf_file.path.display(),
                            cf_file.cf,
                            checksum,
                            cf_file.checksum[i]
                        ),
                    ));
                }
            }

            let tmp_paths = cf_file.tmp_file_paths();
            let paths = cf_file.file_paths();
            for (i, tmp_path) in tmp_paths.iter().enumerate() {
                file_system::rename(tmp_path, &paths[i])?;

View on GitHub (pinned to 78aedc1c81)

Solutions

Remove the corrupt snapshot files and let TiKV request a fresh snapshot from the leader.
Run hardware health checks (SMART, memtest) on the affected node; the error often indicates real disk/memory corruption.
Confirm the sender's data is intact (its own checksums/replication are healthy) and retry the transfer.
Check for external processes or encryption-at-rest key managers interfering with the file bytes.

Example fix

# before
# keep restarting with the same corrupt snap -> repeated checksum error
# after
rm -rf /path/to/tikv/snap/<region_dir>  # purge corrupt snapshot
systemctl restart tikv                   # node re-requests a clean snapshot

Defensive patterns

Strategy: fallback

Validate before calling

let expected = cf_file.checksum[i];
let actual = crc32_of_file(&cf_file.path)?;
if actual != expected {
    // discard file and re-request the snapshot
}

Try / catch

match finalize_result {
    Err(e) if e.kind() == std::io::ErrorKind::InvalidData
        && e.to_string().contains("checksum mismatches") => {
        // purge corrupt snapshot, verify hardware, re-request from leader
    }
    other => other?,
}

Prevention

Run periodic SMART/memory diagnostics on TiKV nodes.
Use ECC RAM and end-to-end checksumming storage.
Never modify snapshot files on disk after receipt.
Track corruption rates via TiKV metrics; investigate any nonzero trend.

When it happens

Trigger: Calling the snapshot receive/finalize path where `file_for_recving.write_digest.finalize() != cf_file.checksum[i]`, i.e. the file contents' checksum differs from the meta-declared checksum.

Common situations: Bit-flip corruption in transit or on disk (failing RAM/NVMe); a file rewritten after the checksum was computed; mixing snapshot metas and files from different snapshots; storage-layer bugs altering bytes.

Related errors

AI-assisted analysis of tikv/tikv@78aedc1c81 (2026-09-03). Data as JSON: /api/errors/ddccd5625b8d3589. Report an issue: GitHub.