dotnet/orleans · warning · OrleansException

SuspectingSilos.Length of {0} as read from Azure table is no

Error message

SuspectingSilos.Length of {0} as read from Azure table is not equal to SuspectingTimes.Length of {1}

What it means

Thrown by AzureBasedMembershipTable.Parse when the SuspectingSilos and SuspectingTimes pipe-delimited lists have different lengths. Membership uses these parallel lists to record which silos voted a peer as suspect and when; a length mismatch indicates a corrupt or partially-written entity. OrleansException (not ArgumentException) is thrown inside Parse and, because Parse is called within the per-entry try, is logged-and-ignored via LogErrorParsingMembershipTableDataIgnoring, so the offending entry is skipped rather than fatal.

Source

Thrown at src/Azure/Orleans.Clustering.AzureStorage/AzureBasedMembershipTable.cs:276

            if (!string.IsNullOrEmpty(tableEntry.SuspectingSilos))
            {
                string[] silos = tableEntry.SuspectingSilos.Split('|');
                foreach (string silo in silos)
                {
                    suspectingSilos.Add(SiloAddress.FromParsableString(silo));
                }
            }

            if (!string.IsNullOrEmpty(tableEntry.SuspectingTimes))
            {
                string[] times = tableEntry.SuspectingTimes.Split('|');
                foreach (string time in times)
                    suspectingTimes.Add(LogFormatter.ParseDate(time));
            }

            if (suspectingSilos.Count != suspectingTimes.Count)
                throw new OrleansException(string.Format("SuspectingSilos.Length of {0} as read from Azure table is not equal to SuspectingTimes.Length of {1}", suspectingSilos.Count, suspectingTimes.Count));

            for (int i = 0; i < suspectingSilos.Count; i++)
                parse.AddSuspector(suspectingSilos[i], suspectingTimes[i]);

            return parse;
        }

        private static SiloInstanceTableEntry Convert(MembershipEntry memEntry, string deploymentId)
        {
            var tableEntry = new SiloInstanceTableEntry
            {
                DeploymentId = deploymentId,
                Address = memEntry.SiloAddress.Endpoint.Address.ToString(),
                Port = memEntry.SiloAddress.Endpoint.Port.ToString(CultureInfo.InvariantCulture),
                Generation = memEntry.SiloAddress.Generation.ToString(CultureInfo.InvariantCulture),
                HostName = memEntry.HostName,
                Status = memEntry.Status.ToString(),
                ProxyPort = memEntry.ProxyPort.ToString(CultureInfo.InvariantCulture),

View on GitHub (pinned to fca799fa70)

Solutions

  1. Inspect the offending entity's SuspectingSilos and SuspectingTimes fields and reconcile them (trim the longer list to match).
  2. Let the table self-heal: because the entry is skipped, the next membership update will overwrite the suspector lists consistently.
  3. Force a membership refresh (restart a silo or call CleanupDefunctSiloEntries) to re-stamp suspect data.
  4. Avoid manual edits to suspector fields; they are written atomically by Orleans in normal operation.

Example fix

// remediation in Azure Storage Explorer
// entity RowKey=<silo>: set SuspectingSilos and SuspectingTimes to '' (empty)
// then let Orleans repopulate them on the next suspicion cycle
Defensive patterns

Strategy: try-catch

Validate before calling

// validate suspector parallel lists before relying on them:
int silos = string.IsNullOrEmpty(entry.SuspectingSilos) ? 0 : entry.SuspectingSilos.Split('|').Length;
int times = string.IsNullOrEmpty(entry.SuspectingTimes) ? 0 : entry.SuspectingTimes.Split('|').Length;
if (silos != times) { /* skip or repair entry */ }

Try / catch

// Parse swallows this via LogErrorParsingMembershipTableDataIgnoring, so the entry is skipped.
// To detect: subscribe to the LogErrorParsingMembershipTableDataIgnoring log event and alert on it.

Prevention

When it happens

Trigger: A membership table entity has SuspectingSilos with N entries and SuspectingTimes with M != N entries (split on '|'). Occurs when a silo crashed mid-write of those two fields, or when an external tool edited only one field.

Common situations: Silo crash between updating SuspectingSilos and SuspectingTimes; manual entity edits; storage serialization that dropped one field; mixed-version cluster writing fields with different separators.

Related errors


AI-assisted analysis of dotnet/orleans@fca799fa70 (2026-08-13). Data as JSON: /api/errors/857ea89ec7401b30. Report an issue: GitHub.