{"record":{"id":"e63a5fb11ec52acc","repo":"aeron-io/aeron","slug":"failed-to-start-clustered-service-s","errorCode":null,"errorMessage":"failed to start clustered service(s)","messagePattern":"failed to start clustered service\\(s\\)","errorType":"exception","errorClass":"AgentTerminationException","httpStatus":null,"severity":"critical","filePath":"aeron-cluster/src/main/java/io/aeron/cluster/ConsensusModuleAgent.java","lineNumber":3526,"sourceCode":"            if (!recoveryPlan.snapshots().isEmpty())\n            {\n                loadSnapshot(recoveryPlan.snapshots().get(0), archive);\n            }\n            else if (null != consensusModuleExtension)\n            {\n                consensusModuleExtension.onStart(this, null);\n            }\n\n            idleStrategy.reset();\n            while (!ServiceAck.hasReached(expectedAckPosition, serviceAckId, serviceAckQueues))\n            {\n                idle(consensusModuleAdapter.poll());\n            }\n\n            final ServiceAck[] serviceAcks = ServiceAck.pollServiceAcks(serviceAckQueues);\n            if (!ServiceAck.areAllRelevantIdsNonNull(\"failed to start clustered service\", serviceAcks, ctx.errorLog()))\n            {\n                throw new AgentTerminationException(\"failed to start clustered service(s)\");\n            }\n\n            captureServiceClientIds(serviceAcks);\n            ++serviceAckId;\n        }\n\n        return recoveryPlan;\n    }\n\n    private RecordingLog.RecoveryPlan recoverFromBootstrapState()\n    {\n        final ConsensusModuleStateExport bootstrapState = ctx.bootstrapState();\n\n        logRecordingId(bootstrapState.logRecordingId);\n        final RecordingLog.RecoveryPlan recoveryPlan = recordingLog.createRecoveryPlan(\n            archive, serviceCount, logRecordingId);\n\n        expectedAckPosition = bootstrapState.expectedAckPosition;","sourceCodeStart":3508,"sourceCodeEnd":3544,"githubUrl":"https://github.com/aeron-io/aeron/blob/6d60124e15e35c11b49ba2e3c2c2858a09a18803/aeron-cluster/src/main/java/io/aeron/cluster/ConsensusModuleAgent.java#L3508-L3544","documentation":"During startup the consensus module waits for each clustered service's initial ack. If polled ServiceAcks contain null relevantIds (service never sent its initial ack), AgentTerminationException is thrown because the cluster cannot start without all services reporting readiness.","triggerScenarios":"Thrown when ServiceAck.areAllRelevantIdsNonNull fails after polling serviceAckQueues during cluster start, i.e. at least one service did not deliver its initial ack (ackId 0) in time.","commonSituations":"A service container crashed on startup (class not found, init exception); wrong service count or serviceId configuration; services blocked in their onStart; version mismatch between cluster and service containers.","solutions":["Inspect ctx.errorLog() written by areAllRelevantIdsNonNull for which service failed","Check each clustered service's own logs for startup exceptions in onStart","Verify serviceCount and serviceId configuration match the deployed containers","Ensure service containers run the same Aeron version as the consensus module; restart the cluster"],"exampleFix":"// before: service crashing in onStart so its ack never arrives\npublic void onStart(Cluster cluster, Image snapshotImage) { throw new RuntimeException(...); }\n\n// after: fail fast with a clear message and correct startup\npublic void onStart(Cluster cluster, Image snapshotImage)\n{\n    // validate dependencies before marking ready\n    this.store = openStore(); // throws descriptive error if unavailable\n}","handlingStrategy":"validation","validationCode":"// before starting the cluster, ensure all services are resolvable\nfor (ClusteredService service : services)\n{\n    Objects.requireNonNull(service, \"null ClusteredService in configuration\");\n}","typeGuard":null,"tryCatchPattern":"catch (AgentTerminationException e)\n{\n    if (e.getMessage().contains(\"failed to start clustered service\"))\n    {\n        // consult ctx.errorLog() written by ServiceAck.areAllRelevantIdsNonNull\n        log.error(\"a service never acked startup; check service container logs\", e);\n    }\n    throw e;\n}","preventionTips":["Test each service's onStart in isolation before cluster launch","Match serviceCount and serviceIds across cluster and containers","Pin the same Aeron version in cluster and service containers","Add startup timeouts and logging inside service onStart implementations"],"tags":["aeron","cluster","startup","service-ack","termination"],"backgroundTag":"module-init-failed","analyzedSha":"6d60124e15e35c11b49ba2e3c2c2858a09a18803","analyzedAt":"2026-09-12T11:17:07.683Z","contentChangedAt":"2026-09-12T11:17:07.683Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}