dotnet/orleans · error · KeyNotFoundException

Could not find table version row or found too many entries.

Error message

Could not find table version row or found too many entries. Was looking for key {0}, found = {1}

What it means

Thrown by OrleansSiloInstanceManager.FindSiloEntryAndTableVersionRow when the Azure Table query for a silo row plus the version row returns fewer than 1 or more than 2 entities. A correct read yields exactly the silo row and the version row; anything else means the silo is not registered or the partition is inconsistent. KeyNotFoundException includes the requested siloAddress and the rows actually found.

Source

Thrown at src/Azure/Orleans.Clustering.AzureStorage/OrleansSiloInstanceManager.cs:253

            else
            {
                var tasks = new List<Task>();
                foreach (var batch in entriesList.BatchIEnumerable(this.storagePolicyOptions.MaxBulkUpdateRows))
                {
                    tasks.Add(storage.DeleteTableEntriesAsync(batch));
                }
                await Task.WhenAll(tasks);
            }
        }

        internal async Task<List<(SiloInstanceTableEntry, string)>> FindSiloEntryAndTableVersionRow(SiloAddress siloAddress)
        {
            string rowKey = SiloInstanceTableEntry.ConstructRowKey(siloAddress);

            var filter = TableClient.CreateQueryFilter($"(PartitionKey eq {DeploymentId}) and ((RowKey eq {rowKey}) or (RowKey eq {SiloInstanceTableEntry.TABLE_VERSION_ROW}))");
            var queryResults = await storage.ReadTableEntriesAndEtagsAsync(filter);
            if (queryResults.Count < 1 || queryResults.Count > 2)
                throw new KeyNotFoundException(string.Format("Could not find table version row or found too many entries. Was looking for key {0}, found = {1}", siloAddress, Utils.EnumerableToString(queryResults)));

            var numTableVersionRows = 0;
            foreach (var entry in queryResults)
            {
                if (entry.Item1.RowKey == SiloInstanceTableEntry.TABLE_VERSION_ROW)
                {
                    numTableVersionRows++;
                }
            }

            if (numTableVersionRows < 1)
                throw new KeyNotFoundException(string.Format("Did not read table version row. Read = {0}", Utils.EnumerableToString(queryResults)));

            if (numTableVersionRows > 1)
                throw new KeyNotFoundException(string.Format("Read {0} table version rows, while was expecting only 1. Read = {1}", numTableVersionRows, Utils.EnumerableToString(queryResults)));

            return queryResults;
        }

View on GitHub (pinned to fca799fa70)

Solutions

1. Confirm the silo has registered: wait for RegisterSiloInstance to complete before reading, or retry with backoff.
2. Verify the DeploymentId matches the cluster the silo belongs to.
3. Inspect the partition for duplicate/corrupt rows and remove extras.
4. Handle KeyNotFoundException gracefully if the lookup is best-effort (e.g. gossip), rather than treating it as fatal.

Example fix

// before
var rows = await manager.FindSiloEntryAndTableVersionRow(addr); // throws if not registered

// after
try { var rows = await manager.FindSiloEntryAndTableVersionRow(addr); }
catch (KeyNotFoundException) { /* silo not yet registered; retry or skip */ }

// before reading a specific silo, confirm it is registered:
var entries = await manager.FindAllSiloEntries();
bool registered = entries.Any(e => e.Entity.RowKey == SiloInstanceTableEntry.ConstructRowKey(addr));
if (!registered) { /* skip or wait */ }

try { var rows = await manager.FindSiloEntryAndTableVersionRow(addr); }
catch (KeyNotFoundException) { /* silo not registered or partition inconsistent; retry or skip */ }

When it happens

Trigger: Reading a specific silo's membership entry when the silo has not yet registered (count 0), or when duplicate/version anomalies produced >2 rows. The query filters PartitionKey=DeploymentId AND (RowKey=silo OR RowKey=Version).

Common situations: Querying a silo before it has inserted its row (startup race); the silo was deleted/unregistered but another silo still references it; duplicated silo rows from a retry storm; partition key mismatch (wrong DeploymentId).

Related errors

• Did not read table version row. Read = {0}
• The table version row does not contain a membership version.
• The membership table does not contain a version row.
• SuspectingSilos.Length of {0} as read from Azure table is no
• The table version entry must have a membership version.

AI-assisted analysis of dotnet/orleans@fca799fa70 (2026-08-13). Data as JSON: /api/errors/e35753e68c1394e9. Report an issue: GitHub.