{"record":{"id":"353f8213bbbf9d43","repo":"microsoft/garnet","slug":"failed-to-validate-main-store-metadata-at-insertio","errorCode":null,"errorMessage":"Failed to validate main store metadata at insertion","messagePattern":"Failed to validate main store metadata at insertion","errorType":"exception","errorClass":"GarnetException","httpStatus":null,"severity":"warning","filePath":"libs/cluster/Server/Replication/CheckpointStore.cs","lineNumber":148,"sourceCode":"            else\n            {\n                // We don't have multiple writers because this method is called under the CheckpointLock\n                // So it is safe to update in-place.\n                tail.next = entry;\n                tail = tail.next;\n            }\n\n            logger?.LogCheckpointEntry(LogLevel.Trace, nameof(AddCheckpointEntry), entry);\n\n            if (safelyRemoveOutdated)\n                DeleteOutdatedCheckpoints();\n\n            bool ValidateCheckpointEntry(CheckpointEntry entry)\n            {\n                try\n                {\n                    if (!clusterProvider.replicationManager.TryAcquireSettledMetadataForMainStore(entry, out _, out _))\n                        throw new GarnetException(\"Failed to validate main store metadata at insertion\");\n\n                    return true;\n                }\n                catch (Exception ex)\n                {\n                    logger?.LogCheckpointEntry(LogLevel.Error, ex.Message, entry);\n                    return false;\n                }\n            }\n        }\n\n        /// <summary>\n        /// Method used to delete outdated checkpoints from in-memory list of checkpoint entries\n        /// </summary>\n        private void DeleteOutdatedCheckpoints()\n        {\n            // If only one in-memory checkpoint return\n            if (head == tail) return;","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/microsoft/garnet/blob/951b0fc6838721f89d102c2bbe1b914e8d39d700/libs/cluster/Server/Replication/CheckpointStore.cs#L130-L166","documentation":"Raised inside AddCheckpointEntry.ValidateCheckpointEntry when replicationManager.TryAcquireSettledMetadataForMainStore fails for the entry being inserted. Crucially it is thrown inside a local try/catch that logs the message at Error level and returns false, so the exception does not propagate to the caller; the visible symptom is a logged error and a failed validation rather than a thrown GarnetException.","triggerScenarios":"Inserting a checkpoint entry whose main-store settled metadata cannot be acquired/verified by the replication manager (e.g. metadata not yet settled, store not initialized, replication manager stopped).","commonSituations":"Checkpoint completion racing with replication shutdown; a replica attempting checkpoint insertion before its main-store metadata has settled; a corrupted or missing metadata token after recovery.","solutions":["Check logs for this Error-level message and inspect whether the replication manager was healthy at insertion time.","Ensure checkpoint insertion happens after main-store metadata has settled (replication manager fully initialized).","If it recurs, verify TryAcquireSettledMetadataForMainStore's preconditions (store open, metadata present) are met before AddCheckpointEntry."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Ensure replication manager is ready before checkpoint insertion\nif (!clusterProvider.replicationManager.TryAcquireSettledMetadataForMainStore(entry, out _, out _))\n    logger?.LogError(\"Skipping checkpoint insertion: main-store metadata not settled for {entry}\", entry);\nelse\n    store.AddCheckpointEntry(entry, fullCheckpoint);","typeGuard":null,"tryCatchPattern":"// The exception is already caught internally and logged; monitor logs rather than catching\nif (!added)\n{\n    logger?.LogWarning(\"Checkpoint entry validation failed; investigate replication manager state\");\n}","preventionTips":["Monitor logs for the Error-level validation message as the primary signal.","Ensure replication manager is fully initialized before checkpoint insertion.","Avoid inserting checkpoints during replication shutdown/restart windows."],"tags":["cluster","replication","checkpoint","validation","logging","garnet"],"backgroundTag":null,"analyzedSha":"951b0fc6838721f89d102c2bbe1b914e8d39d700","analyzedAt":"2026-08-13T19:01:32.939Z","schemaVersion":2},"datasetVersion":"2026-08-13T19:17:28.613Z"}