microsoft/FASTER · error · FasterException

Unable to set last valid segment to

Error message

Unable to set last valid segment to {lastValidSegment}, last available segment on disk is {lastAvailSegment}

What it means

The mirror of the first-valid-segment check: after recovery scan, the checkpoint's last valid segment must not exceed the newest segment physically on disk. If the checkpoint claims a lastValidSegment beyond what the device holds, the tail of the log is missing and recovery would produce an incomplete/corrupt state, so it throws. Guarding against truncated or partially deleted log tails.

Solutions

Restore the complete log files (all segments up to lastValidSegment) from backup before recovering.
Recover from an older checkpoint whose lastValidSegment exists on the device.
Attach the original device containing the full segment range instead of a partial copy.

Example fix

// before
// log dir contains only segments 0-3, checkpoint expects segments 0-7
store.Recover(checkpointSettings); // throws

// after
// restore segments 4-7 from backup, or recover from an older checkpoint:
var target = latestCheckpointWhoseLastSegmentExistsOnDisk();
store.Recover(target);

Defensive patterns

Strategy: try-catch

Validate before calling

// confirm all segments up to the checkpoint tail exist on the device before recovery

Try / catch

try { store.Recover(checkpointSettings); }
catch (FasterException ex) when (ex.Message.Contains("last valid segment")) {
    // restore missing tail segments from backup or pick an older checkpoint
}

Prevention

Back up/restore the complete log directory atomically with checkpoints.
Verify file counts/sizes after copying log files between machines.
Monitor disk-full conditions that could truncate tail writes.

When it happens

Trigger: Recovering a checkpoint whose lastValidSegment is greater than the last segment found on the attached LogDevice - e.g. tail log files deleted, disk full during writes lost data, or a partial checkpoint copy.

Common situations: Incomplete backup/restore of the log directory; a device that failed mid-write losing tail segments; copying only part of the checkpoint/log files to another machine.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/c66d98dad301d271. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/Allocator/AllocatorBase.cs:1120

            var lastAvailSegment = device.EndSegment;

            if (FlushedUntilAddress > GetFirstValidLogicalAddress(0))
            {
                int currTailSegment = (int)(FlushedUntilAddress >> LogSegmentSizeBits);
                if ((FlushedUntilAddress & ((1L << LogSegmentSizeBits) - 1)) == 0)
                    currTailSegment--;

                if (currTailSegment > lastAvailSegment)
                    lastAvailSegment = currTailSegment;
            }

            logger?.LogInformation("Available segment range on device: [{firstAvailSegment}--{lastAvailSegment}]", firstAvailSegment, lastAvailSegment);

            if (firstValidSegment < firstAvailSegment)
                throw new FasterException($"Unable to set first valid segment to {firstValidSegment}, first available segment on disk is {firstAvailSegment}");

            if (lastAvailSegment >= 0 && lastValidSegment > lastAvailSegment)
                throw new FasterException($"Unable to set last valid segment to {lastValidSegment}, last available segment on disk is {lastAvailSegment}");

            if (trimLog)
            {
                logger?.LogInformation("Trimming disk segments until (not including) {firstSegment}", firstValidSegment);
                TruncateUntilAddressBlocking(firstValidSegment << LogSegmentSizeBits);

                for (int s = lastValidSegment + 1; s <= lastAvailSegment; s++)
                {
                    logger?.LogInformation("Trimming tail segment {s} on disk", s);
                    RemoveSegment(s);
                }
            }
        }

        /// <summary>
        /// Initialize allocator
        /// </summary>
        /// <param name="firstValidAddress"></param>

View on GitHub (pinned to 321d872eab)