dotnet/orleans · error · InvalidOperationException

Index {indexName} in table {tableName} has failed to reach t

Error message

Index {indexName} in table {tableName} has failed to reach the desired status of {desiredStatus}

What it means

Thrown by TableIndexWaitOnStatusAsync after polling a Global Secondary Index when desiredStatus was requested but the index either could not be found or ended in a status other than desired (typically ACTIVE). The loop waits while the GSI is in whileStatus, then asserts the final IndexStatus equals desiredStatus; a FAILED or missing index trips the guard.

Source

Thrown at src/AWS/Shared/Storage/DynamoDBStorage.cs:452

        private async Task<TableDescription> TableIndexWaitOnStatusAsync(string tableName, string indexName, IndexStatus whileStatus, IndexStatus? desiredStatus = null, int delay = 2000, CancellationToken cancellationToken = default)
        {
            TableDescription ret;
            GlobalSecondaryIndexDescription? index = null;

            do
            {
                if (index != null)
                {
                    await Task.Delay(delay, cancellationToken);
                }

                ret = (await GetTableDescription(tableName, cancellationToken))!;
                index = ret.GlobalSecondaryIndexes?.Find(index => index.IndexName == indexName);
            } while (index != null && index.IndexStatus == whileStatus);

            if (desiredStatus != null && (index == null || index.IndexStatus != desiredStatus))
            {
                throw new InvalidOperationException($"Index {indexName} in table {tableName} has failed to reach the desired status of {desiredStatus}");
            }

            return ret;
        }

        /// <summary>
        /// Delete a table from DynamoDB
        /// </summary>
        /// <param name="tableName">The name of the table to delete</param>
        /// <returns></returns>
        public Task DeleTableAsync(string tableName)
        {
            try
            {
                return _ddbClient.DeleteTableAsync(new DeleteTableRequest { TableName = tableName });
            }
            catch (Exception exc)
            {

View on GitHub (pinned to fca799fa70)

Solutions

1. Run DescribeTable and inspect GlobalSecondaryIndexes for the failing index name and its IndexStatus.
2. Fix capacity/quota issues (raise write capacity on the table during backfill, or switch to PAY_PER_REQUEST) and redeploy so Init re-attempts.
3. If the index was removed intentionally, update the provider's secondary-index configuration so it no longer expects it.
4. For transient cases, confirm the index is ACTIVE in the AWS console and retry silo startup.

Example fix

// before: GSI backfill fails under low provisioned write capacity
opt.UseProvisionedThroughput = true;
opt.WriteCapacityUnits = 5; // too low for GSI backfill

// after: allow enough capacity for backfill (or on-demand)
opt.UseProvisionedThroughput = false; // PAY_PER_REQUEST during deployment
// then revert to provisioned once the GSI is ACTIVE if desired

// Pre-flight: confirm GSIs exist and are ACTIVE before relying on them
var desc = await DescribeExistingTableAsync(tableName);
var gsi = desc?.GlobalSecondaryIndexes?.Find(i => i.IndexName == expectedIndex);
if (gsi is null || gsi.IndexStatus != IndexStatus.ACTIVE)
    _logger.LogWarning("GSI {Index} is {Status}; init may fail", expectedIndex, gsi?.IndexStatus);

static bool IsIndexActive(GlobalSecondaryIndexDescription? i) =>
    i is not null && i.IndexStatus == IndexStatus.ACTIVE;

try { await silo.StartAsync(); }
catch (InvalidOperationException ix) when (ix.Message.Contains("has failed to reach the desired status"))
{
    _logger.LogCritical("DynamoDB GSI did not reach ACTIVE; raise capacity or remove the index expectation and retry");
    throw;
}

When it happens

Trigger: Produced during provider init when initializing a table that defines a GSI and waiting for it to become ACTIVE. Triggered by the GSI creation failing (e.g., not enough capacity for backfilling), the index being deleted out-of-band, or DescribeTable no longer reporting the index.

Common situations: A GSI backfill running out of provisioned capacity and failing; an operator dropping the index mid-deploy; misconfiguration of secondary indexes between deployments; eventual-consistency returning a transiently missing index.

Related errors

• Table {tableName} has failed to reach the desired status of
• Configuration for DynamoDBGrainStorage {name} is invalid. Ta
• Configuration for DynamoDBGrainStorage {name} is invalid. Re
• Configuration for DynamoDBGrainStorage {name} is invalid. Wr
• Data too large to write to DynamoDB table. Size={dataSize} M

AI-assisted analysis of dotnet/orleans@fca799fa70 (2026-08-13). Data as JSON: /api/errors/3689bab0e126d8f1. Report an issue: GitHub.