microsoft/FASTER · error · FasterException

Can compact only until Log.SafeReadOnlyAddress

Error message

Can compact only until Log.SafeReadOnlyAddress

What it means

Lookup-based compaction can only compact records that are no longer mutable, i.e. below Log.SafeReadOnlyAddress. Passing a untilAddress greater than SafeReadOnlyAddress would race with in-flight writes, so FASTER throws.

Solutions

1. Clamp untilAddress to hlog.SafeReadOnlyAddress before calling Compact.
2. Use Log.ShiftReadOnlyAddress() to advance SafeReadOnlyAddress if appropriate, then compact.
3. Pause or quiesce writers, or track a known-durable address from a checkpoint to use as untilAddress.

Example fix

// before
fht.Compact<Input, Output, Context, Functions, CF>(functions, cf, CompactionType.Lookup, ref input, ref output, hlog.TailAddress);
// after
long untilAddress = hlog.SafeReadOnlyAddress;
fht.Compact<Input, Output, Context, Functions, CF>(functions, cf, CompactionType.Lookup, ref input, ref output, untilAddress);

long untilAddress = Math.Min(requestedUntilAddress, fht.Log.SafeReadOnlyAddress);

When it happens

Trigger: Calling fht.Compact<...>(..., untilAddress: X, ...) with CompactionType.Lookup where X > hlog.SafeReadOnlyAddress.

Common situations: Compacting with untilAddress set to hlog.TailAddress or a computed address while concurrent writes/reads are shifting the read-only boundary; compacting immediately after heavy inserts.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Invalid compaction type
• Out of order message within session
• Unexpected status of SubscribeKV
• Cannot use BlittableParameterSerializer with non-blittable…
• The inner list is full!

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/cd59351085005434. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/Compaction/FASTERCompaction.cs:40

        /// <returns>Address until which compaction was done</returns>
        internal long Compact<Input, Output, Context, Functions, CompactionFunctions>(Functions functions, CompactionFunctions cf, ref Input input, ref Output output, long untilAddress, CompactionType compactionType, SessionVariableLengthStructSettings<Value, Input> sessionVariableLengthStructSettings = null)
            where Functions : IFunctions<Key, Value, Input, Output, Context>
            where CompactionFunctions : ICompactionFunctions<Key, Value>
        {
            return compactionType switch
            {
                CompactionType.Scan => CompactScan<Input, Output, Context, Functions, CompactionFunctions>(functions, cf, ref input, ref output, untilAddress, sessionVariableLengthStructSettings),
                CompactionType.Lookup => CompactLookup<Input, Output, Context, Functions, CompactionFunctions>(functions, cf, ref input, ref output, untilAddress, sessionVariableLengthStructSettings),
                _ => throw new FasterException("Invalid compaction type"),
            };
        }

        private long CompactLookup<Input, Output, Context, Functions, CompactionFunctions>(Functions functions, CompactionFunctions cf, ref Input input, ref Output output, long untilAddress, SessionVariableLengthStructSettings<Value, Input> sessionVariableLengthStructSettings)
            where Functions : IFunctions<Key, Value, Input, Output, Context>
            where CompactionFunctions : ICompactionFunctions<Key, Value>
        {
            if (untilAddress > hlog.SafeReadOnlyAddress)
                throw new FasterException("Can compact only until Log.SafeReadOnlyAddress");

            var lf = new LogCompactionFunctions<Key, Value, Input, Output, Context, Functions>(functions);
            using var fhtSession = For(lf).NewSession<LogCompactionFunctions<Key, Value, Input, Output, Context, Functions>>(sessionVariableLengthStructSettings: sessionVariableLengthStructSettings);

            using (var iter1 = Log.Scan(Log.BeginAddress, untilAddress))
            {
                long numPending = 0;
                while (iter1.GetNext(out var recordInfo))
                {
                    ref var key = ref iter1.GetKey();
                    ref var value = ref iter1.GetValue();

                    if (!recordInfo.Tombstone && !cf.IsDeleted(ref key, ref value))
                    {
                        var status = fhtSession.CompactionCopyToTail(ref key, ref input, ref value, ref output, iter1.NextAddress);
                        if (status.IsPending && ++numPending > 256)
                        {
                            fhtSession.CompletePending(wait: true);

View on GitHub (pinned to 321d872eab)