{"record":{"id":"d69ea5cbe90958fa","repo":"nats-io/nats-server","slug":"message-sequence-d-out-of-order","errorCode":null,"errorMessage":"message sequence %d out of order","messagePattern":"message sequence (.+?) out of order","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"server/stream_backup.go","lineNumber":516,"sourceCode":"\t\t\tif err := checkUsageLimits(); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\t\tbuf, err := io.ReadAll(tr)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to read message sequence %d: %w\", seq, err)\n\t\t}\n\t\tif hdr.HeaderSize > int64(len(buf)) {\n\t\t\treturn nil, fmt.Errorf(\"failed to parse message sequence %d: invalid header length\", seq)\n\t\t}\n\t\tif int64(len(buf)) != declaredSize {\n\t\t\treturn nil, fmt.Errorf(\"failed to read message sequence %d: unexpected payload size\", seq)\n\t\t}\n\t\tsubj := hdr.Name\n\t\tmhdr := buf[:hdr.HeaderSize]\n\t\tmsg := buf[hdr.HeaderSize : hdr.HeaderSize+hdr.PayloadSize]\n\t\tif seq <= lseq {\n\t\t\treturn nil, fmt.Errorf(\"message sequence %d out of order\", seq)\n\t\t}\n\t\t// We could have deleted messages since the last message we stored, if so\n\t\t// we should work out what the gap is and skip those sequences.\n\t\tif gap := seq - lseq - 1; gap > 0 {\n\t\t\tif err := store.SkipMsgs(lseq+1, gap); err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"failed to process gap: %w\", err)\n\t\t\t}\n\t\t}\n\t\tlseq = seq\n\t\tttl, err := getMessageTTL(mhdr)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to parse message TTL: %w\", err)\n\t\t}\n\t\thdrTime := time.Unix(0, hdr.Timestamp)\n\t\tif ttl > 0 && time.Now().After(hdrTime.Add(time.Duration(ttl)*time.Second)) {\n\t\t\t// If the TTL has exceeded then there isn't much point in storing the message,\n\t\t\t// but we still need to preserve the sequence.\n\t\t\tif err := store.SkipMsgs(seq, 1); err != nil {","sourceCodeStart":498,"sourceCodeEnd":534,"githubUrl":"https://github.com/nats-io/nats-server/blob/3a66a489d262bf89b71a71c955c94920394532f3/server/stream_backup.go#L498-L534","documentation":"RestoreStreamV2 enforces that message sequences in the backup are strictly increasing and contiguous. This error is returned when a message's sequence (seq) is less than or equal to the last stored sequence (lseq), meaning the backup contains out-of-order or duplicate sequence data that would break stream state invariants.","triggerScenarios":"A backup block carries a seq <= lseq, i.e. duplicate or decreasing sequences; caused by corrupt backup data, wrongly concatenated backups, or replaying blocks from an older snapshot into a newer one.","commonSituations":"Merging two backup files, resuming an interrupted backup incorrectly, or restoring a corrupted archive where block boundaries were lost and records misparsed.","solutions":["Restore from a single, complete backup file produced in one session; do not concatenate backups","Verify the archive integrity (checksum) and re-create the backup if corrupt","Check that the backup was not appended/merged across multiple snapshots","If resuming incremental backups, use the server-supported mechanisms rather than manual merging"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Never merge or append separate backup archives; verify a single archive\nif isMergedArchive(path) { return fmt.Errorf(\"refusing to restore merged/concatenated backup\") }","typeGuard":null,"tryCatchPattern":"if err := restore(...); err != nil && strings.Contains(err.Error(), \"out of order\") {\n    // mark backup invalid; obtain a new full backup\n}","preventionTips":["Produce each backup in a single uninterrupted session","Do not concatenate snapshots or resume via manual appends","Verify archive integrity before restore","Use incremental backup features supported by the server instead of manual merges"],"tags":["jetstream","restore","sequences","corruption"],"backgroundTag":"corrupt-backup-file","analyzedSha":"3a66a489d262bf89b71a71c955c94920394532f3","analyzedAt":"2026-09-02T04:41:54.247Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}