GoogleCloudPlatform/microservices-demo · critical · RpcException

FAILED_PRECONDITION

FAILED_PRECONDITION

Error message

Unable to access cart storage due to an internal error. {ex}

What it means

AlloyDBCartStore.AddItemAsync wraps every AlloyDB (Npgsql) operation in a try/catch and rethrows any exception as a gRPC RpcException with status FailedPrecondition and the message 'Unable to access cart storage due to an internal error. {ex}'. It means the cart service could not complete its SQL insert/upsert of an item into the user's cart — usually a connectivity, schema, or database availability problem, not a bad request.

Source

Thrown at src/cartservice/src/cartstore/AlloyDBCartStore.cs:98

            // Use INSERT ... ON CONFLICT to prevent duplicate key error
            var insertCmd = $@"
                INSERT INTO {tableName} (userId, productId, quantity)
                VALUES ('{userId}', '{productId}', {totalQuantity})
                ON CONFLICT (userId, productId)
                DO UPDATE SET quantity = {totalQuantity};
            ";

            await using (var cmdInsert = dataSource.CreateCommand(insertCmd))
            {
                await Task.Run(() =>
                {
                    return cmdInsert.ExecuteNonQueryAsync();
                });
            }
        }
        catch (Exception ex)
        {   
            throw new RpcException(
                new Status(StatusCode.FailedPrecondition, $"Unable to access cart storage due to an internal error. {ex}"));
        }
    }


        public async Task<Hipstershop.Cart> GetCartAsync(string userId)
        {
            Console.WriteLine($"GetCartAsync called for userId={userId}");
            Hipstershop.Cart cart = new();
            cart.UserId = userId;
            try
            {
                await using var dataSource = NpgsqlDataSource.Create(connectionString);

                var cartFetchCmd = $"SELECT productId, quantity FROM {tableName} WHERE userId = '{userId}'";
                var cmd = dataSource.CreateCommand(cartFetchCmd);
                await using (var reader = await cmd.ExecuteReaderAsync())
                {

View on GitHub (pinned to 72ba613a05)

Solutions

1. Read the inner exception text after the message in the log — it names the real Npgsql cause (connection refused, relation does not exist, auth failure)
2. Verify the AlloyDB connection string/env vars and that the DB host is reachable from the service (try psql or a TCP check from the same network)
3. Run the schema initialization (create carts table) if the error mentions relation/table not found
4. Restart/recycle the pod to clear a poisoned connection pool, and check AlloyDB instance health/monitoring

Example fix

// before (typical env misconfig)
CARTSERVICE_DB_URI=alloydb://user:pass@10.0.0.1:5432/cartdb   # wrong host/private IP
// after
CARTSERVICE_DB_URI=alloydb://user:pass@<alloydb-instance-ip>:5432/cartdb  # reachable + schema initialized

// Before AddItemAsync, verify storage health
var store = new AlloyDBCartStore(connectionString);
if (!store.Ping()) {
    throw new InvalidOperationException("Cart storage (AlloyDB) is not reachable; check connection string and schema.");
}

bool IsFailedPreconditionStorage(RpcException ex) =>
    ex.StatusCode == StatusCode.FailedPrecondition &&
    ex.Status.Detail.StartsWith("Unable to access cart storage");

try {
    await cartStore.AddItemAsync(userId, productId, quantity);
} catch (RpcException ex) when (ex.StatusCode == StatusCode.FailedPrecondition) {
    logger.LogError(ex, "Cart storage write failed for user {UserId}", userId);
    throw new ApplicationException("Cart is temporarily unavailable. Please try again.", ex);
}

When it happens

Trigger: AddItemAsync(userId, item, productId, quantity) fails on its INSERT ... ON CONFLICT upsert into the carts table: connection refused/timeouts to AlloyDB, missing or uninitialized schema (table 'carts' does not exist), wrong credentials, or pool exhaustion.

Common situations: AlloyDB instance not provisioned or private-IP unreachable from the cart service pod; CARTSERVICE DB env vars (connection string) misconfigured; schema init script never run; VPC/network auth (IAM authdb) misconfigured after deployment changes.

Related errors

• FAILED_PRECONDITION
• FAILED_PRECONDITION
• InvalidCreditCard
• UnacceptedCreditCard
• ExpiredCreditCard

AI-assisted analysis of GoogleCloudPlatform/microservices-demo@72ba613a05 (2026-09-02). Data as JSON: /api/errors/6b56d3454decd15f. Report an issue: GitHub.