{"record":{"id":"a0cdcfa8017d3423","repo":"dotnet/orleans","slug":"one-or-more-errors-occurred","errorCode":null,"errorMessage":"One or more errors occurred.","messagePattern":"One or more errors occurred\\.","errorType":"exception","errorClass":"AggregateException","httpStatus":null,"severity":"error","filePath":"src/Azure/Orleans.Streaming.EventHubs/Providers/Streams/EventHub/EventHubAdapterReceiver.cs","lineNumber":367,"sourceCode":"                this.monitor?.TrackShutdown(true, watch.Elapsed, null);\n            }\n            catch (Exception ex)\n            {\n                watch.Stop();\n                this.monitor?.TrackShutdown(false, watch.Elapsed, ex);\n                throw;\n            }\n\n            static void ThrowIfAny(List<Exception> exceptions)\n            {\n                if (exceptions.Count == 1)\n                {\n                    ExceptionDispatchInfo.Capture(exceptions[0]).Throw();\n                }\n\n                if (exceptions.Count > 1)\n                {\n                    throw new AggregateException(exceptions);\n                }\n            }\n        }\n\n        private static IEventHubReceiver CreateReceiver(EventHubPartitionSettings partitionSettings, string offset, ILogger logger)\n        {\n            return new EventHubReceiverProxy(partitionSettings, offset, logger);\n        }\n\n        /// <summary>\n        /// For test purpose. ConfigureDataGeneratorForStream will configure a data generator for the stream\n        /// </summary>\n        /// <param name=\"streamId\"></param>\n        internal void ConfigureDataGeneratorForStream(StreamId streamId)\n        {\n            (this.receiver as EventHubPartitionGeneratorReceiver)?.ConfigureDataGeneratorForStream(streamId);\n        }\n","sourceCodeStart":349,"sourceCodeEnd":385,"githubUrl":"https://github.com/dotnet/orleans/blob/fca799fa70ecb6ad975224271703ca43221f58de/src/Azure/Orleans.Streaming.EventHubs/Providers/Streams/EventHub/EventHubAdapterReceiver.cs#L349-L385","documentation":"This AggregateException is thrown by EventHubAdapterReceiver.Shutdown when multiple independent components fail during shutdown — the method collects exceptions from checkpoint flushing, receiver closing, cache disposal, and the close-task await into a list, and if more than one exception is collected it wraps them all in an AggregateException rather than reporting only the first. If exactly one exception occurs it is re-thrown directly via ExceptionDispatchInfo preserving the original stack trace.","triggerScenarios":"Calling Shutdown (or the silo stopping and triggering receiver shutdown) when two or more of the following fail simultaneously: (1) checkpointer.FlushAsync throws (e.g., Azure Storage outage), (2) EventHubReceiver.CloseAsync throws (e.g., network failure), (3) cache.Dispose throws (e.g., eviction strategy disposal error), (4) awaiting the close task throws due to timeout or cancellation.","commonSituations":"Silo shutdown during a partial network outage where both Event Hub connections and checkpoint storage (Azure Blob Storage) are unreachable; a timeout is passed to Shutdown that is too short, causing the CancellationTokenSource to cancel both the flush and the close operations; cascading failures where a connection drop affects both the Event Hub receiver and the storage-based checkpointer.","solutions":["Inspect AggregateException.InnerExceptions to identify all root causes — the first may not be the most important.","Ensure Azure Storage (for checkpointing) and Event Hub connectivity are healthy before initiating a graceful shutdown.","If using a timeout for Shutdown, ensure it is generous enough (e.g., 30+ seconds) to allow checkpoint flushing and receiver closing to complete.","If the silo is shutting down due to an error, resolve the underlying connectivity issue first, then retry shutdown."],"exampleFix":"// before\ntry\n{\n    await receiver.Shutdown(TimeSpan.FromSeconds(5));\n}\ncatch (Exception ex)\n{\n    logger.LogError(ex, \"Shutdown failed\"); // loses inner exceptions\n}\n\n// after\ntry\n{\n    await receiver.Shutdown(TimeSpan.FromSeconds(30));\n}\ncatch (AggregateException aggEx)\n{\n    foreach (var inner in aggEx.InnerExceptions)\n        logger.LogError(inner, \"Shutdown sub-operation failed\");\n}\ncatch (Exception ex)\n{\n    logger.LogError(ex, \"Shutdown failed\");\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try\n{\n    await receiver.Shutdown(shutdownTimeout);\n}\ncatch (AggregateException aggEx)\n{\n    // Log all inner exceptions — each represents a different failed sub-operation\n    foreach (var inner in aggEx.InnerExceptions)\n    {\n        logger.LogWarning(inner, \"Shutdown sub-operation failed for partition {Partition}\", partition);\n    }\n    // Decide whether to rethrow based on your error policy\n}\ncatch (Exception ex) when (ex is not AggregateException)\n{\n    // Single exception — re-thrown via ExceptionDispatchInfo, preserves original stack\n    logger.LogError(ex, \"Shutdown failed for partition {Partition}\", partition);\n    throw;\n}","preventionTips":["Use a generous shutdown timeout (30+ seconds) to allow checkpoint flushing and receiver closing to complete under load.","Ensure Azure Storage (checkpointing) and Event Hub connectivity are healthy before graceful shutdown.","Monitor for partial connectivity issues (e.g., storage throttling) that can cause cascading shutdown failures.","Always catch AggregateException and inspect InnerExceptions, not just the outer Message."],"tags":["shutdown","aggregate-exception","eventhub","receiver","error-handling","network","checkpoint"],"backgroundTag":null,"analyzedSha":"fca799fa70ecb6ad975224271703ca43221f58de","analyzedAt":"2026-08-13T19:55:57.938Z","schemaVersion":2},"datasetVersion":"2026-08-14T00:17:13.853Z"}