GoogleCloudPlatform/microservices-demo · critical · RpcException

FAILED_PRECONDITION

FAILED_PRECONDITION

Error message

Can't access cart storage at {databaseString}. {ex}

What it means

SpannerCartStore.AddItemAsync wraps its Spanner read/write transaction (upserting cart items) in a try/catch that rethrows any SpannerException as an RpcException with StatusCode.FailedPrecondition and message 'Can't access cart storage at {databaseString}. {ex}'. The message includes the Spanner database string, indicating the cart item could not be persisted to Cloud Spanner.

Source

Thrown at src/cartservice/src/cartstore/SpannerCartStore.cs:99

                        new SpannerParameterCollection
                        {
                            { "userId", SpannerDbType.String },
                            { "productId", SpannerDbType.String },
                            { "quantity", SpannerDbType.Int64 }
                        });
                    cmd.Parameters["userId"].Value = userId;
                    cmd.Parameters["productId"].Value = productId;
                    cmd.Parameters["quantity"].Value = currentQuantity + quantity;
                    cmd.Transaction = transaction;
                    await Task.Run(() =>
                    {
                        return cmd.ExecuteNonQueryAsync();
                    });
                });
            }
            catch (Exception ex)
            {
                throw new RpcException(
                    new Status(StatusCode.FailedPrecondition, $"Can't access cart storage at {databaseString}. {ex}"));
            }
        }


        public async Task<Hipstershop.Cart> GetCartAsync(string userId)
        {
            Console.WriteLine($"GetCartAsync called for userId={userId}");
            Hipstershop.Cart cart = new();
            try
            {
                using SpannerConnection spannerConnection = new(databaseString);
                var cmd = spannerConnection.CreateSelectCommand(
                    $"SELECT * FROM {TableName} WHERE userId = @userId",
                    new SpannerParameterCollection {
                        { "userId", SpannerDbType.String }
                    }
                );

View on GitHub (pinned to 72ba613a05)

Solutions

1. Check the inner SpannerException for the precise cause (NotFound for missing DB/table, PermissionDenied for IAM, DeadlineExceeded)
2. Verify SPANNER_PROJECT, SPANNER_INSTANCE, and SPANNER_DATABASE env vars match an existing database
3. Confirm the carts table schema was applied and the service account has the required Spanner IAM roles
4. Check Google Cloud Spanner monitoring for incidents and retry once transient unavailability clears

Example fix

// before
SPANNER_PROJECT=wrong-proj SPANNER_INSTANCE=nonexistent SPANNER_DATABASE=nodb
// after
SPANNER_PROJECT=my-project SPANNER_INSTANCE=cart-instance SPANNER_DATABASE=cart-db  # table 'carts' initialized

// Validate Spanner configuration before cart operations
var project = Environment.GetEnvironmentVariable("SPANNER_PROJECT");
var instance = Environment.GetEnvironmentVariable("SPANNER_INSTANCE");
var database = Environment.GetEnvironmentVariable("SPANNER_DATABASE");
if (string.IsNullOrEmpty(project) || string.IsNullOrEmpty(instance) || string.IsNullOrEmpty(database)) {
    throw new InvalidOperationException("SPANNER_PROJECT/INSTANCE/DATABASE must be set and point to an initialized carts database.");
}
if (!cartStore.Ping()) {
    throw new InvalidOperationException("Cannot reach Spanner database; check IAM roles and schema.");
}

bool IsSpannerCartFailure(RpcException ex) =>
    ex.StatusCode == StatusCode.FailedPrecondition &&
    ex.Status.Detail.StartsWith("Can't access cart storage at");

try {
    await cartStore.AddItemAsync(userId, productId, quantity);
} catch (RpcException ex) when (IsSpannerCartFailure(ex)) {
    logger.LogError(ex, "Spanner cart write failed for {UserId}", userId);
    throw new ApplicationException("Cart storage is temporarily unavailable; please retry.", ex);
}

When it happens

Trigger: AddItemAsync fails inside its session/transaction on Spanner: SPANNER_PROJECT/SPANNER_INSTANCE/SPANNER_DATABASE env vars wrong, database or table missing, IAM permission denied, or transient Spanner unavailability/timeouts.

Common situations: Spanner instance not provisioned or schema (carts table) never created in a new environment; service account lacking spanserver.data/database roles; env vars pointing to a deleted or different project; regional endpoint/egress issues.

Related errors

• FAILED_PRECONDITION
• FAILED_PRECONDITION
• InvalidCreditCard
• UnacceptedCreditCard
• ExpiredCreditCard

AI-assisted analysis of GoogleCloudPlatform/microservices-demo@72ba613a05 (2026-09-02). Data as JSON: /api/errors/bae9871f2b1fb6e0. Report an issue: GitHub.