microsoft/garnet · error · InvalidDataException

Advancing to next segment exceeds maximum object log segment

Error message

Advancing to next segment exceeds maximum object log segment.

What it means

Thrown by ObjectLogFilePositionInfo.AdvanceToNextSegment when incrementing the current segment id by one would exceed MaxSegmentId. This is the segment-at-a-time variant of the capacity check: the object-log file position has reached the last addressable segment and cannot open another.

Source

Thrown at libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/ObjectLogFilePositionInfo.cs:157

            }

            // Note: If size == remaining, we will advance to the start of the next segment.
            size -= remaining;

            // Move to the next segment(s).
            long nextSegmentId = SegmentId + (int)(size / SegmentSize) + 1;
            if (nextSegmentId > MaxSegmentId)
                throw new InvalidDataException($"Advancing position by {size:N} bytes exceeds maximum object log segment.");

            SegmentId = (int)nextSegmentId;
            Offset += size & (SegmentSize - 1);
        }

        public void AdvanceToNextSegment()
        {
            long nextSegmentId = SegmentId + 1;
            if (nextSegmentId > MaxSegmentId)
                throw new InvalidDataException($"Advancing to next segment exceeds maximum object log segment.");
            SegmentId = (int)nextSegmentId;
            Offset = 0;
        }

        public readonly ulong CurrentAddress => ((ulong)SegmentId << SegmentSizeBits) | Offset;

        public static ulong operator -(ObjectLogFilePositionInfo left, ObjectLogFilePositionInfo right)
        {
            Debug.Assert(left.SegmentSizeBits == right.SegmentSizeBits, "Segment size bits must match to compute distance");
            Debug.Assert((left.word & SegmentAndOffsetMask) >= (right.word & SegmentAndOffsetMask), "comparison position must be greater");
            var segmentDiff = (ulong)(left.SegmentId - right.SegmentId);
            if (segmentDiff == 0)
                return left.Offset - right.Offset;
            return ((segmentDiff - 1) * left.SegmentSize) + (left.SegmentSize - right.Offset) + left.Offset;
        }

        public readonly ulong SegmentSize => 1UL << SegmentSizeBits;

View on GitHub (pinned to 951b0fc683)

Solutions

  1. Lower ObjectLogSegmentSizeBits (within [22,62]) to increase the number of segment ids and total capacity.
  2. Recycle/compact the object log if the store supports it, reducing the live segment range.
  3. Investigate abnormally fast segment growth (e.g. repeated large-object writes).

Example fix

// before
logSettings.ObjectLogSegmentSizeBits = 58; // few segment ids
// after
logSettings.ObjectLogSegmentSizeBits = 33; // default, many segment ids
Defensive patterns

Strategy: validation

Validate before calling

var maxSegments = 1L << (64 - logSettings.ObjectLogSegmentSizeBits);
if (maxSegments < 1024) logger.LogWarning("Few object-log segment ids available; consider lowering ObjectLogSegmentSizeBits");

Try / catch

try { /* flush/write that may cross segment boundary */ }
catch (InvalidDataException ex) when (ex.Message.Contains("exceeds maximum object log segment"))
{
    logger.LogError(ex, "Segment id exhausted during segment advance");
    throw;
}

Prevention

When it happens

Trigger: Object-log writing/flushing that fills segments until SegmentId == MaxSegmentId, then needs a new segment (segment boundary reached). Triggered on segment rollover during WriteRecordObjects / OnBufferComplete flush paths.

Common situations: Long-lived store that has written enough object-log data to exhaust segment ids; large ObjectLogSegmentSizeBits (fewer, bigger segments) accelerating id exhaustion; a runaway write loop.

Related errors

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/066da62c3b7e6038. Report an issue: GitHub.