{"record":{"id":"241c9dddbb8e7bea","repo":"microsoft/garnet","slug":"unexpected-end-of-snapshot-object-log-data-while-c","errorCode":null,"errorMessage":"Unexpected end of snapshot object-log data while copying objects during recovery","messagePattern":"Unexpected end of snapshot object-log data while copying objects during recovery","errorType":"exception","errorClass":"TsavoriteException","httpStatus":null,"severity":"critical","filePath":"libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/ObjectLogWriter.cs","lineNumber":147,"sourceCode":"        /// positioned at the record (via <see cref=\"CircularDiskReadBuffer.OnBeginRecord\"/>).\n        /// </summary>\n        /// <param name=\"reader\">The reader over the snapshot object-log, positioned at the record to copy.</param>\n        /// <param name=\"totalLength\">The total number of object-log bytes for the record (key plus value).</param>\n        public void CopyRecoveredObjectBytes(ObjectLogReader<TStoreFunctions> reader, ulong totalLength)\n        {\n            if (totalLength > 0)\n            {\n                var buffer = flushBuffers.bufferPool.Get(IStreamBuffer.BufferSize);\n                try\n                {\n                    var chunkSpan = buffer.TotalValidSpan;\n                    var remaining = totalLength;\n                    while (remaining > 0)\n                    {\n                        var requestLength = (int)Math.Min(remaining, (ulong)chunkSpan.Length);\n                        var bytesRead = reader.Read(chunkSpan.Slice(0, requestLength));\n                        if (bytesRead == 0)\n                            throw new TsavoriteException(\"Unexpected end of snapshot object-log data while copying objects during recovery\");\n                        Write(chunkSpan.Slice(0, bytesRead));\n                        remaining -= (ulong)bytesRead;\n                    }\n                }\n                finally\n                {\n                    flushBuffers.bufferPool.Return(buffer);\n                }\n            }\n\n            // Signal completion, as WriteRecordObjects does.\n            flushBuffers.OnRecordComplete();\n        }\n\n        /// <summary>Start off the write using the full span of the <see cref=\"OverflowByteArray\"/>.</summary>\n        /// <param name=\"overflow\">The <see cref=\"OverflowByteArray\"/> to write.</param>\n        void WriteDirect(OverflowByteArray overflow) => WriteDirect(overflow, overflow.ReadOnlySpan, refCountedGCHandle: default);\n","sourceCodeStart":129,"sourceCodeEnd":165,"githubUrl":"https://github.com/microsoft/garnet/blob/951b0fc6838721f89d102c2bbe1b914e8d39d700/libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/ObjectLogWriter.cs#L129-L165","documentation":"Thrown by ObjectLogWriter.CopyRecoveredObjectBytes when copying object-log bytes from the snapshot object reader during recovery and reader.Read returns 0 (EOF) before the expected totalLength bytes are consumed. The snapshot object-log stream is shorter than the main-log record says it should be — the object-log snapshot is truncated or mismatched with the main log.","triggerScenarios":"Recovery object-log copy where the snapshot object-log device ends prematurely relative to the copyObjectLength the main record references. Triggered in the recovery loop after CreateSnapshotObjectReader positions the reader.","commonSituations":"Truncated or partially restored object-log snapshot file; snapshot of object log taken from a different/older checkpoint than the main log; object-log device write that didn't flush fully before crash; corruption cutting the file short.","solutions":["Restore the complete object-log snapshot from the same checkpoint as the main-log snapshot.","Verify the object-log snapshot file size is at least as large as the highest referenced position before recovery.","Recover from an earlier consistent checkpoint if the object-log snapshot is unusable.","Ensure durable flush of the object-log device on checkpoint to avoid a short file after a crash."],"exampleFix":"// before: recover with a truncated object-log snapshot\nstore.Recover();\n// after: restore the full matching object-log snapshot device, then recover\nlogSettings.ObjectLogDevice = Devices.CreateLogDevice(fullObjectLogPath);\nstore.Recover();","handlingStrategy":"validation","validationCode":"// Verify the object-log snapshot length covers the highest referenced position before Recover\nvar fi = new FileInfo(objectLogSnapshotPath);\nif (!fi.Exists || fi.Length < maxReferencedObjectPosition)\n    throw new InvalidOperationException(\"Object-log snapshot missing or truncated\");","typeGuard":null,"tryCatchPattern":"try { store.Recover(); }\ncatch (TsavoriteException ex) when (ex.Message.Contains(\"Unexpected end of snapshot object-log data\"))\n{\n    logger.LogError(ex, \"Object-log snapshot truncated; restore complete checkpoint and retry\");\n    throw;\n}","preventionTips":["Restore the complete object-log snapshot alongside the main-log snapshot.","Ensure durable flush of the object-log device on every checkpoint.","Validate snapshot file sizes before starting recovery."],"tags":["recovery","object-log","snapshot","truncation"],"backgroundTag":null,"analyzedSha":"951b0fc6838721f89d102c2bbe1b914e8d39d700","analyzedAt":"2026-08-13T19:01:32.939Z","schemaVersion":2},"datasetVersion":"2026-08-13T19:17:28.613Z"}