microsoft/FASTER · error · FasterException

Invalid index commit metadata for ID

Error message

Invalid index commit metadata for ID

What it means

IndexCheckpointMetadata.Recover throws when checkpointManager.GetIndexCheckpointMetadata(guid) returns null, i.e., no index checkpoint metadata exists for the given token. This usually means the token is invalid, the checkpoint was deleted, or the recovery call was made before a matching checkpoint was taken.

Solutions

  1. Verify the Guid matches an existing index checkpoint in the checkpoint manager's store
  2. Take a full (index + hybrid-log) checkpoint so both tokens exist, then recover with the full-checkpoint token
  3. Check checkpoint retention/purge policy isn't deleting the checkpoint before recovery
  4. Confirm you're pointing at the same checkpoint directory/container used at checkpoint time

Example fix

// before: recovering index from a hybrid-log-only checkpoint token
fasterKV.Recover(hybridLogToken);
// after: use the token from TakeFullCheckpointAsync / GetIndexCheckpointToken
(long token, _) = await fasterKV.TakeFullCheckpointAsync();
fasterKV.Recover(new Guid(token.ToString()));
Defensive patterns

Strategy: try-catch

Validate before calling

// Ensure the token came from a full checkpoint or an index checkpoint specifically
if (token == default || !knownIndexCheckpointTokens.Contains(token))
    throw new InvalidOperationException($"No index checkpoint known for token {token}");

Try / catch

try { fasterKV.Recover(token); }
catch (FasterException ex) when (ex.Message.StartsWith("Invalid index commit metadata for ID"))
{
    // token is missing: enumerate available checkpoints and fail fast with a clear message
    throw new InvalidOperationException($"Index checkpoint {token} not found; check retention policy and checkpoint directory");
}

Prevention

When it happens

Trigger: Calling Recover with a Guid token that does not exist in the checkpoint store: token from an index checkpoint passed where a hybrid-log checkpoint is expected, checkpoint already purged by retention/cleanup, or recovery attempted before any checkpoint completed.

Common situations: Persisting the wrong token (e.g., storing _hybridLogCheckpointToken but calling index recovery); log/checkpoint retention deleting old checkpoints; environment mismatch (recovery against a different container's checkpoint store); taking only a hybrid-log checkpoint but trying to recover the index too.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/5edf6c1878d248d9. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/Index/Common/Contexts.cs:825

            value = reader.ReadLine();
            startLogicalAddress = long.Parse(value);

            value = reader.ReadLine();
            finalLogicalAddress = long.Parse(value);

            if (cversion != CheckpointVersion)
                throw new FasterException("Invalid version");

            if (checksum != Checksum())
                throw new FasterException("Invalid checksum for checkpoint");
        }

        public void Recover(Guid guid, ICheckpointManager checkpointManager)
        {
            this.token = guid;
            var metadata = checkpointManager.GetIndexCheckpointMetadata(guid);
            if (metadata == null)
                throw new FasterException("Invalid index commit metadata for ID " + guid.ToString());
            using (StreamReader s = new(new MemoryStream(metadata)))
                Initialize(s);
        }

        public readonly byte[] ToByteArray()
        {
            using (MemoryStream ms = new())
            {
                using (StreamWriter writer = new(ms))
                {
                    writer.WriteLine(CheckpointVersion); // checkpoint version
                    writer.WriteLine(Checksum()); // checksum

                    writer.WriteLine(token);
                    writer.WriteLine(table_size);
                    writer.WriteLine(num_ht_bytes);
                    writer.WriteLine(num_ofb_bytes);
                    writer.WriteLine(num_buckets);

View on GitHub (pinned to 321d872eab)