microsoft/garnet · error · TsavoriteException

ReadBuffers are required to ReadRecordObjects

Error message

ReadBuffers are required to ReadRecordObjects

What it means

Thrown by ObjectLogReader.ReadRecordObjects when readBuffers is null. ReadRecordObjects reads overflow keys/values and object values from the object log, which requires a populated read-buffer set (CircularDiskReadBuffer). A null readBuffers means the reader was constructed/entered for a code path that doesn't supply object-log read buffers, yet the record claims to have objects.

Source

Thrown at libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/ObjectLogReader.cs:98

        ///     If we don't have <paramref name="requestedKey"/>, this is either ReadAtAddress (which is an implicit match) or Scan or Restore.</item>
        /// <item>If we have an Overflow or Object value, read and store it in the transient <see cref="ObjectIdMap"/> in <paramref name="logRecord"/>.</item>
        /// </list>
        /// </summary>
        /// <param name="logRecord">The initial record read from disk from Pending IO, so it is of size <see cref="IStreamBuffer.DefaultInitialIORecordSize"/> or less.</param>
        /// <param name="requestedKey">The requested key, if not ReadAtAddress; we will compare to see if it matches the record.</param>
        /// <param name="segmentSizeBits">Number of bits in segment size</param>
        /// <returns>False if requestedKey is set and we read an Overflow key and it did not match; otherwise true</returns>
        [MethodImpl(MethodImplOptions.NoInlining)]
        public bool ReadRecordObjects<TKey>(ref LogRecord logRecord, TKey requestedKey, int segmentSizeBits)
            where TKey : IKey
#if NET9_0_OR_GREATER
                , allows ref struct
#endif
        {
            Debug.Assert(logRecord.DataHeader.RecordHasObjects, "Inline records should have been checked by the caller");
            Debug.Assert(logRecord.HasReuseObjectIdForSize, "ReadRecordObjects requires the ReuseObjectIdForSize flag to be set on the ObjectLogPosition");
            if (readBuffers is null)
                throw new TsavoriteException("ReadBuffers are required to ReadRecordObjects");

            // R11 encoding: lengths returned by GetObjectLogRecordStartPositionAndLengths combine the RDH low bits with the next 32 bits
            // from the int* slot at keyAddress/valueAddress. No length prefix is in the object stream.
            var positionWord = logRecord.GetObjectLogRecordStartPositionAndLengths(out var keyLength, out var valueLength);
            if (!readBuffers.OnBeginRecord(new ObjectLogFilePositionInfo(positionWord, segmentSizeBits)))
                throw new TsavoriteException("ReadRecordObjects found no data available in ReadBuffers");

            // TODO: Optimize the reading of large internal sector-aligned parts of Overflow Keys and Values to read directly into the overflow, similar to how ObjectLogWriter writes
            //       directly from overflow. This requires changing the read-ahead in CircularDiskReadBuffer.OnBeginReadRecords and the "backfill" in CircularDiskReadBuffer.MoveToNextBuffer.

            // Note: Similar logic to this is in DiskLogRecord.Deserialize.
            var keyWasSet = false;
            try
            {
                if (logRecord.DataHeader.KeyIsOverflow)
                {
                    // This assignment also allocates the slot in ObjectIdMap, overwriting the int* slot at keyAddress
                    // (which held the high 32 bits of the on-disk key length per R11). The raw RDH KeyLength is restored

View on GitHub (pinned to 951b0fc683)

Solutions

1. Ensure the object-log read path always allocates readBuffers (configure NumberOfDeserializationBuffers correctly for the object allocator).
2. Only call ReadRecordObjects on readers initialized with a read buffer; for inline-only paths, skip object reading.
3. Check logRecord.DataHeader.RecordHasObjects before invoking the object-read path.

Example fix

// before
reader.ReadRecordObjects(ref logRecord, requestedKey, segmentSizeBits); // readBuffers null
// after
if (logRecord.DataHeader.RecordHasObjects && reader.HasReadBuffers)
    reader.ReadRecordObjects(ref logRecord, requestedKey, segmentSizeBits);

if (logRecord.DataHeader.RecordHasObjects && !reader.HasReadBuffers)
    throw new InvalidOperationException("Record has objects but no read buffers are configured");

static bool CanReadObjects(ObjectLogReader r, LogRecord rec) => rec.DataHeader.RecordHasObjects && r.HasReadBuffers;

When it happens

Trigger: Invoking ReadRecordObjects on an ObjectLogReader/DiskStreamReadBuffer whose readBuffers field was never set — e.g. a single-record disk IO path or a configuration where NumberOfDeserializationBuffers-related buffers weren't allocated, but the record has RecordHasObjects.

Common situations: A read path that was set up for inline-only records but received a record with objects; a refactor that dropped buffer initialization; misconfigured object store where the object-log read buffer is absent.

Related errors

• ReadRecordObjects found no data available in ReadBuffers
• {nameof(settings.LogSettings.NumberOfFlushBuffers)} must be
• {nameof(settings.LogSettings.NumberOfDeserializationBuffers)
• {nameof(settings.LogSettings.ObjectLogSegmentSizeBits)} must
• Exceeded maximum response size of ({Array.MaxLength:N0}) byt

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/3e9ee602ed1863ec. Report an issue: GitHub.