{"record":{"id":"c5ba65162f77a757","repo":"nats-io/nats-server","slug":"snapshot-check-detected-d-bad-messages","errorCode":null,"errorMessage":"snapshot check detected %d bad messages","messagePattern":"snapshot check detected (.+?) bad messages","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"server/filestore.go","lineNumber":12919,"sourceCode":"\t}\n\t// Mark us as snapshotting\n\tfs.sips += 1\n\tfs.mu.Unlock()\n\n\tif checkMsgs {\n\t\tld, err := fs.checkMsgs()\n\t\tclearSips := func() {\n\t\t\tfs.mu.Lock()\n\t\t\tfs.sips--\n\t\t\tfs.mu.Unlock()\n\t\t}\n\t\tif err != nil {\n\t\t\tclearSips()\n\t\t\treturn nil, fmt.Errorf(\"snapshot check failed: %w\", err)\n\t\t}\n\t\tif ld != nil && len(ld.Msgs) > 0 {\n\t\t\tclearSips()\n\t\t\treturn nil, fmt.Errorf(\"snapshot check detected %d bad messages\", len(ld.Msgs))\n\t\t}\n\t}\n\n\tpr, pw := net.Pipe()\n\n\t// Set a write deadline here to protect ourselves.\n\tif deadline > 0 {\n\t\tpw.SetWriteDeadline(time.Now().Add(deadline))\n\t}\n\n\t// We can add to our stream while snapshotting but not \"user\" delete anything.\n\tvar state StreamState\n\tfs.FastState(&state)\n\n\t// Stream in separate Go routine.\n\terrCh := make(chan error, 1)\n\tgo fs.streamSnapshot(pw, includeConsumers, errCh)\n","sourceCodeStart":12901,"sourceCodeEnd":12937,"githubUrl":"https://github.com/nats-io/nats-server/blob/3a66a489d262bf89b71a71c955c94920394532f3/server/filestore.go#L12901-L12937","documentation":"Raised when a snapshot consistency check completes but finds messages: the verification read (ld) contains bad messages, so the snapshot is rejected rather than producing a corrupt backup. The count of bad messages is included in the message. This indicates data problems in the stream's file storage rather than an API misuse.","triggerScenarios":"Stream snapshot verification where the check read returns non-empty ld.Msgs after verifying snapshot consistency, meaning messages read back differ from what the snapshot state expects.","commonSituations":"Streams affected by prior disk corruption, crashed writes, or files tampered with/synced inconsistently (e.g. rsync of a live stream directory); running an older server on data written by a newer version.","solutions":["Treat the stream as suspect: validate/repair by restoring the stream from a known-good backup.","Check server logs and the wrapped errors around this check for the specific bad sequences.","Avoid copying live filestore directories; use proper snapshot/backup APIs.","If corruption is hardware-driven, replace/repair storage and rebuild the stream."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Go: on bad-message detection, fall back to a known-good backup\nif strings.Contains(err.Error(), \"snapshot check detected\") {\n    log.Printf(\"stream data integrity compromised: %v\", err)\n    // restore stream from last good snapshot\n}","preventionTips":["Only snapshot via official APIs, never copy live data files","Schedule integrity checks on streams","Retain multiple backup generations","Investigate any prior crash or hardware errors on the storage host"],"tags":["jetstream","filestore","snapshot","data-integrity"],"backgroundTag":"snapshot-bad-messages","analyzedSha":"3a66a489d262bf89b71a71c955c94920394532f3","analyzedAt":"2026-09-02T04:41:54.247Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}