microsoft/FASTER · error · Exception

Metadata of size does not fit in delta log space of size

Error message

Metadata of size {commitMetadata.Length} does not fit in delta log space of size {length}

What it means

During CommitLogIncrementalCheckpoint, the checkpoint metadata is serialized and an attempt is made to write it into a single delta-log record. If the serialized metadata length exceeds the space returned by deltaLog.Allocate, the manager seals the entry and throws. This means the incremental checkpoint metadata cannot fit in the allocated delta log entry, so the incremental checkpoint cannot proceed.

Solutions

1. Take a full (non-incremental) checkpoint instead, which writes metadata through a path not bounded by a single delta entry.
2. Increase the delta log entry/segment size so metadata of the current size fits.
3. Reduce metadata size (e.g. fewer iterators, smaller cookie, compact recovery info) before checkpointing.
4. Update/upgrade the library if a fixed larger default delta space is available.

Example fix

// before
manager.CommitLogIncrementalCheckpoint(token); // may throw if metadata too large

// after
// fall back to a full checkpoint when incremental fails
try {
    manager.CommitLogIncrementalCheckpoint(token);
} catch (Exception) {
    manager.CommitLogCheckpoint(token); // full checkpoint path
}

try { manager.CommitLogIncrementalCheckpoint(token); } catch (Exception ex) when (ex.Message.Contains("does not fit in delta log space")) { manager.CommitLogCheckpoint(token); }

When it happens

Trigger: Calling CommitLogIncrementalCheckpoint when commitMetadata.Length > the length returned by deltaLog.Allocate(out length, out physicalAddress); typically metadata grew beyond the expected delta record size.

Common situations: Very large numbers of partitions/segments or huge cookie blobs inflating metadata; misconfigured delta log entry size; running incremental checkpoints with metadata that only fits a full checkpoint.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• Unexpected entry type
• Invalid log commit metadata for ID
• The inner list is full!
• Unable to set first valid segment to
• Unable to set last valid segment to

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/edbd86bfee00a652. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/Index/CheckpointManagement/DeviceLogCommitCheckpointManager.cs:266

                logTokenHistory[logTokenHistoryOffset] = logToken;
                logTokenHistoryOffset = (byte)((logTokenHistoryOffset + 1) % logTokenCount);
                if (prior != default)
                    deviceFactory.Delete(checkpointNamingScheme.LogCheckpointBase(prior));
            }
        }

        /// <inheritdoc />
        public virtual unsafe void CommitLogIncrementalCheckpoint(Guid logToken, long version, byte[] commitMetadata, DeltaLog deltaLog)
        {
            deltaLog.Allocate(out int length, out long physicalAddress);
            if (length < commitMetadata.Length)
            {
                deltaLog.Seal(0, DeltaLogEntryType.CHECKPOINT_METADATA);
                deltaLog.Allocate(out length, out physicalAddress);
                if (length < commitMetadata.Length)
                {
                    deltaLog.Seal(0);
                    throw new Exception($"Metadata of size {commitMetadata.Length} does not fit in delta log space of size {length}");
                }
            }
            fixed (byte* ptr = commitMetadata)
            {
                Buffer.MemoryCopy(ptr, (void*)physicalAddress, commitMetadata.Length, commitMetadata.Length);
            }
            deltaLog.Seal(commitMetadata.Length, DeltaLogEntryType.CHECKPOINT_METADATA);
            deltaLog.FlushAsync().Wait();
        }

        /// <inheritdoc />
        public IEnumerable<Guid> GetLogCheckpointTokens()
        {
            return deviceFactory.ListContents(checkpointNamingScheme.LogCheckpointBasePath()).Select(e => checkpointNamingScheme.Token(e));
        }

        /// <inheritdoc />
        public virtual byte[] GetLogCheckpointMetadata(Guid logToken, DeltaLog deltaLog, bool scanDelta, long recoverTo)

View on GitHub (pinned to 321d872eab)