{"record":{"id":"ddccd5625b8d3589","repo":"tikv/tikv","slug":"snapshot-file-for-cf-checksum-mismatches-re","errorCode":null,"errorMessage":"snapshot file {} for cf {} checksum mismatches, real checksum {}, expected checksum {}","messagePattern":"snapshot file (.+?) for cf (.+?) checksum mismatches, real checksum (.+?), expected checksum (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"components/raftstore/src/store/snap.rs","lineNumber":1272,"sourceCode":"                file_for_recving.file.sync_all()?;\n\n                if file_for_recving.written_size != cf_file.size[i] {\n                    return Err(io::Error::new(\n                        ErrorKind::InvalidData,\n                        format!(\n                            \"snapshot file {} for cf {} size mismatches, \\\n                            real size {}, expected size {}\",\n                            cf_file.path.display(),\n                            cf_file.cf,\n                            file_for_recving.written_size,\n                            cf_file.size[i]\n                        ),\n                    ));\n                }\n\n                let checksum = file_for_recving.write_digest.finalize();\n                if checksum != cf_file.checksum[i] {\n                    return Err(io::Error::new(\n                        ErrorKind::InvalidData,\n                        format!(\n                            \"snapshot file {} for cf {} checksum \\\n                            mismatches, real checksum {}, expected \\\n                            checksum {}\",\n                            cf_file.path.display(),\n                            cf_file.cf,\n                            checksum,\n                            cf_file.checksum[i]\n                        ),\n                    ));\n                }\n            }\n\n            let tmp_paths = cf_file.tmp_file_paths();\n            let paths = cf_file.file_paths();\n            for (i, tmp_path) in tmp_paths.iter().enumerate() {\n                file_system::rename(tmp_path, &paths[i])?;","sourceCodeStart":1254,"sourceCodeEnd":1290,"githubUrl":"https://github.com/tikv/tikv/blob/78aedc1c81ef3f7d8bacc6e9d09f56460f134937/components/raftstore/src/store/snap.rs#L1254-L1290","documentation":"Immediately after the size check, the same snapshot finalization loop in components/raftstore/src/store/snap.rs:1272 compares each CF file's finalized CRC (from write_digest) with the checksum recorded in the snapshot meta. On mismatch it returns InvalidData: \"snapshot file {} for cf {} checksum mismatches, real checksum {}, expected checksum {}\". The content matches the declared size but the bytes differ — silent corruption is being rejected.","triggerScenarios":"Calling the snapshot receive/finalize path where `file_for_recving.write_digest.finalize() != cf_file.checksum[i]`, i.e. the file contents' checksum differs from the meta-declared checksum.","commonSituations":"Bit-flip corruption in transit or on disk (failing RAM/NVMe); a file rewritten after the checksum was computed; mixing snapshot metas and files from different snapshots; storage-layer bugs altering bytes.","solutions":["Remove the corrupt snapshot files and let TiKV request a fresh snapshot from the leader.","Run hardware health checks (SMART, memtest) on the affected node; the error often indicates real disk/memory corruption.","Confirm the sender's data is intact (its own checksums/replication are healthy) and retry the transfer.","Check for external processes or encryption-at-rest key managers interfering with the file bytes."],"exampleFix":"# before\n# keep restarting with the same corrupt snap -> repeated checksum error\n# after\nrm -rf /path/to/tikv/snap/<region_dir>  # purge corrupt snapshot\nsystemctl restart tikv                   # node re-requests a clean snapshot","handlingStrategy":"fallback","validationCode":"let expected = cf_file.checksum[i];\nlet actual = crc32_of_file(&cf_file.path)?;\nif actual != expected {\n    // discard file and re-request the snapshot\n}","typeGuard":null,"tryCatchPattern":"match finalize_result {\n    Err(e) if e.kind() == std::io::ErrorKind::InvalidData\n        && e.to_string().contains(\"checksum mismatches\") => {\n        // purge corrupt snapshot, verify hardware, re-request from leader\n    }\n    other => other?,\n}","preventionTips":["Run periodic SMART/memory diagnostics on TiKV nodes.","Use ECC RAM and end-to-end checksumming storage.","Never modify snapshot files on disk after receipt.","Track corruption rates via TiKV metrics; investigate any nonzero trend."],"tags":["raftstore","snapshot","checksum","data-corruption"],"backgroundTag":"snapshot-checksum-mismatch","analyzedSha":"78aedc1c81ef3f7d8bacc6e9d09f56460f134937","analyzedAt":"2026-09-03T23:31:32.398Z","contentChangedAt":"2026-09-03T23:31:32.398Z","schemaVersion":2},"datasetVersion":"2026-09-11T07:07:21.782Z"}