microsoft/garnet · error · InvalidDataException

Advancing position by {size:N} bytes exceeds maximum object

Error message

Advancing position by {size:N} bytes exceeds maximum object log segment.

What it means

Thrown by ObjectLogFilePositionInfo.Advance when advancing the object-log file position by 'size' bytes would move the segment id past MaxSegmentId. MaxSegmentId is derived from the number of segment-and-offset bits (a 64-bit-ish address word) minus SegmentSizeBits, so the object-log address space is exhausted. This is a structural capacity ceiling, not a transient condition.

Source

Thrown at libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/ObjectLogFilePositionInfo.cs:147

        }

        public void Advance(ulong size)
        {
            // Does it fit in the current segment?
            var remaining = SegmentSize - Offset;
            if (size < remaining)
            {
                Offset += size;
                return;
            }

            // Note: If size == remaining, we will advance to the start of the next segment.
            size -= remaining;

            // Move to the next segment(s).
            long nextSegmentId = SegmentId + (int)(size / SegmentSize) + 1;
            if (nextSegmentId > MaxSegmentId)
                throw new InvalidDataException($"Advancing position by {size:N} bytes exceeds maximum object log segment.");

            SegmentId = (int)nextSegmentId;
            Offset += size & (SegmentSize - 1);
        }

        public void AdvanceToNextSegment()
        {
            long nextSegmentId = SegmentId + 1;
            if (nextSegmentId > MaxSegmentId)
                throw new InvalidDataException($"Advancing to next segment exceeds maximum object log segment.");
            SegmentId = (int)nextSegmentId;
            Offset = 0;
        }

        public readonly ulong CurrentAddress => ((ulong)SegmentId << SegmentSizeBits) | Offset;

        public static ulong operator -(ObjectLogFilePositionInfo left, ObjectLogFilePositionInfo right)
        {

View on GitHub (pinned to 951b0fc683)

Solutions

Reduce ObjectLogSegmentSizeBits so more segment ids fit in the address word (more total capacity), while staying >= 22.
Investigate the source of the oversized write: the {size:N} value in the message shows how many bytes were requested.
Cap individual object sizes well below the segment capacity and chunk very large values.

Example fix

// before
logSettings.ObjectLogSegmentSizeBits = 60; // leaves only a handful of segment ids
// after
logSettings.ObjectLogSegmentSizeBits = 33; // default 8GB segments, many segment ids available

Defensive patterns

Strategy: validation

Validate before calling

// Sanity-check configured segment bits leave usable segment capacity
const int addressBits = 64;
var maxSegments = (1L << (addressBits - logSettings.ObjectLogSegmentSizeBits));
if (maxSegments < 1024) logger.LogWarning("ObjectLogSegmentSizeBits={Bits} leaves only {N} segment ids", logSettings.ObjectLogSegmentSizeBits, maxSegments);

Try / catch

try { /* object-log write */ }
catch (InvalidDataException ex) when (ex.Message.Contains("exceeds maximum object log segment"))
{
    logger.LogError(ex, "Object-log segment capacity exhausted; lower ObjectLogSegmentSizeBits");
    throw;
}

Prevention

Keep ObjectLogSegmentSizeBits at a moderate value (e.g. 33) to maximize segment count.
Cap individual object sizes and reject pathologically large writes upstream.

When it happens

Trigger: Serializing/writing object-log data such that the running file position crosses into a segment beyond the addressable range; typically a runaway write of an enormous object or an incorrectly huge size argument. Also possible if SegmentSizeBits is configured very large, shrinking the number of available segment bits.

Common situations: A corrupt or absurdly large object length being copied through the object log; misconfigured ObjectLogSegmentSizeBits near kMaxSegmentSizeBits (62) leaving almost no segment ids; a bug producing an oversized Advance argument.

Related errors

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/66ede4b457428fb8. Report an issue: GitHub.