microsoft/FASTER · error · FasterException

Unable to set first valid segment to

Error message

Unable to set first valid segment to {firstValidSegment}, first available segment on disk is {firstAvailSegment}

What it means

During recovery from a checkpoint, FASTER scans the log device to find the range of segments actually present on disk. The checkpoint records the first valid segment of the log; if the requested firstValidSegment is earlier than the first segment available on disk, recovery cannot restore that log state (data was truncated or the wrong device is attached), so it throws. This prevents silently recovering a log with a hole at the front.

Solutions

Attach the correct original LogDevice that contains the checkpoint's segments (verify the path/contents).
Recover from a checkpoint whose firstValidSegment exists on the current device (e.g. use the latest valid checkpoint index).
If the log was intentionally truncated, re-open the store without the stale checkpoint instead of recovering it.

Example fix

// before
using var store = new FasterKV<Key, Value>(size, settings); // fresh device path
store.Recover(new CheckpointSettings { CheckpointDir = oldDir }); // checkpoint expects old segments

// after
// attach the same device/checkpoint set the checkpoint was taken with, or:
// start fresh without calling Recover on a checkpoint whose log segments are gone.

Defensive patterns

Strategy: try-catch

Validate before calling

// verify the oldest segment exists before recovering
bool deviceHasSegment(IDevice dev, long segment) =>
    dev.GetFileSize(segment << segmentSizeBits) > 0 || segment == 0;

Try / catch

try { store.Recover(checkpointSettings); }
catch (FasterException ex) when (ex.Message.Contains("first valid segment")) {
    // fall back to an older checkpoint or start fresh
}

Prevention

Keep the log device and checkpoint directories together; never restore one without the other.
Do not manually delete or trim early log segments while checkpoints reference them.
Test disaster-restore procedures so partial restores are caught before production recovery.

When it happens

Trigger: Calling Recover() with a checkpoint whose first valid segment precedes the oldest segment on the attached LogDevice - typically after the log was truncated, a different (empty/new) device was supplied, or checkpoint metadata references a device that no longer holds those segments.

Common situations: Pointing recovery at a fresh/empty log device while loading an old checkpoint; manual deletion or trimming of log files; moving checkpoints between machines with different disk contents.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/0d4ec78e4746736f. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/Allocator/AllocatorBase.cs:1117

            logger?.LogInformation("Recovery requires disk segments in range [{firstSegment}--{tailStartSegment}]", firstValidSegment, lastValidSegment);

            var firstAvailSegment = device.StartSegment;
            var lastAvailSegment = device.EndSegment;

            if (FlushedUntilAddress > GetFirstValidLogicalAddress(0))
            {
                int currTailSegment = (int)(FlushedUntilAddress >> LogSegmentSizeBits);
                if ((FlushedUntilAddress & ((1L << LogSegmentSizeBits) - 1)) == 0)
                    currTailSegment--;

                if (currTailSegment > lastAvailSegment)
                    lastAvailSegment = currTailSegment;
            }

            logger?.LogInformation("Available segment range on device: [{firstAvailSegment}--{lastAvailSegment}]", firstAvailSegment, lastAvailSegment);

            if (firstValidSegment < firstAvailSegment)
                throw new FasterException($"Unable to set first valid segment to {firstValidSegment}, first available segment on disk is {firstAvailSegment}");

            if (lastAvailSegment >= 0 && lastValidSegment > lastAvailSegment)
                throw new FasterException($"Unable to set last valid segment to {lastValidSegment}, last available segment on disk is {lastAvailSegment}");

            if (trimLog)
            {
                logger?.LogInformation("Trimming disk segments until (not including) {firstSegment}", firstValidSegment);
                TruncateUntilAddressBlocking(firstValidSegment << LogSegmentSizeBits);

                for (int s = lastValidSegment + 1; s <= lastAvailSegment; s++)
                {
                    logger?.LogInformation("Trimming tail segment {s} on disk", s);
                    RemoveSegment(s);
                }
            }
        }

        /// <summary>

View on GitHub (pinned to 321d872eab)