microsoft/garnet · error · TsavoriteException

Object serialized size currently at {valueObjectBytesWritten

Q: What causes "Object serialized size currently at {valueObjectBytesWritten} which exceeds max serialization limit of {IHeapObject.MaxSerializedObjectSize}" in microsoft/garnet?

Reduce the serialized size of the value object — chunk it, stream it, or store the bulk out-of-band and keep a reference.

Error message

Object serialized size currently at {valueObjectBytesWritten} which exceeds max serialization limit of {IHeapObject.MaxSerializedObjectSize}

What it means

Thrown by ObjectLogWriter during in-place serialization when the running serialized byte count of a value object reaches IHeapObject.MaxSerializedObjectSize (1 << 40 = 1 TiB). This is a hard ceiling to prevent a single object from consuming an unbounded object-log segment; exceeding it means the object being serialized is pathologically large or serialization is in a loop.

Source

Thrown at libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/ObjectLogWriter.cs:281

                // If it won't all fit in the remaining buffer, write as much as will.
                var requestLength = (uint)(data.Length - dataStart);
                if (requestLength > writeBuffer.RemainingCapacity)
                    requestLength = (uint)writeBuffer.RemainingCapacity;

                // If it won't all fit in the remaining segment, write as much as will.
                if ((ulong)requestLength > segmentRemainingLen)
                    requestLength = (uint)segmentRemainingLen;
                segmentRemainingLen -= requestLength;

                data.Slice(dataStart, (int)requestLength).CopyTo(writeBuffer.memory.TotalValidSpan.Slice(writeBuffer.currentPosition));
                dataStart += (int)requestLength;
                writeBuffer.currentPosition += (int)requestLength;
                if (inSerialize)
                {
                    valueObjectBytesWritten += requestLength;
                    if (valueObjectBytesWritten >= IHeapObject.MaxSerializedObjectSize)
                        throw new TsavoriteException($"Object serialized size currently at {valueObjectBytesWritten} which exceeds max serialization limit of {IHeapObject.MaxSerializedObjectSize}");
                }

                // See if we're at the end of the buffer or segment.
                if (writeBuffer.RemainingCapacity == 0 || segmentRemainingLen == 0)
                    OnBufferComplete();

                if (segmentRemainingLen == 0)
                {
                    flushBuffers.filePosition.AdvanceToNextSegment();
                    segmentRemainingLen = flushBuffers.filePosition.RemainingSizeInSegment;
                }
            }
        }

        /// <summary>At the end of a buffer, do any processing, flush the current buffer, and move to the next buffer. </summary>
        /// <remarks>Called during Serialize().</remarks>
        void OnBufferComplete()
        {

View on GitHub (pinned to 951b0fc683)

Solutions

Reduce the serialized size of the value object — chunk it, stream it, or store the bulk out-of-band and keep a reference.
Audit the IHeapObject.DoSerialize implementation for runaway writes (loops, repeated data).
If genuinely huge data is required, split into multiple records or use the overflow path rather than one serialized object.

Example fix

// before: Serialize writes the entire giant blob
public void DoSerialize(BinaryWriter w) => w.Write(_hugeBlob);
// after: store blob externally and serialize a reference
public void DoSerialize(BinaryWriter w) { w.Write(_blobHandle); }

Defensive patterns

Strategy: validation

Validate before calling

public void DoSerialize(BinaryWriter w)
{
    if (_estimatedSerializedSize > MaxBeforeWrite) // your own sane cap, e.g. 1GB
        throw new InvalidOperationException("Refusing to serialize an oversized object; chunk it instead");
    /* ... */
}

Try / catch

try { valueObject.Serialize(writer); }
catch (TsavoriteException ex) when (ex.Message.Contains("exceeds max serialization limit"))
{
    logger.LogError(ex, "Object too large to serialize; split or externalize its data");
    throw;
}

Prevention

Audit IHeapObject.DoSerialize implementations for runaway writes.
Chunk very large payloads into multiple records instead of one object.
Enforce an application-level size cap well below 1 TiB.

When it happens

Trigger: Serializing an IHeapObject value whose serialized form is >= 1 TiB, accumulated across the OnSerialize loop in WriteRecordObjects. Triggered when inSerialize is true and valueObjectBytesWritten crosses the limit.

Common situations: A user IHeapObject whose Serialize writes far more than expected (e.g. embedding a huge collection or file); a recursive/infinite serialization writing repeated data; a corrupt size field causing an oversized write; legitimately enormous payloads (rare).

Related errors

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/ee808c80c0b0d911. Report an issue: GitHub.