aeron-io/aeron · critical · AgentTerminationException

failed to start clustered service(s)

Error message

failed to start clustered service(s)

What it means

During startup the consensus module waits for each clustered service's initial ack. If polled ServiceAcks contain null relevantIds (service never sent its initial ack), AgentTerminationException is thrown because the cluster cannot start without all services reporting readiness.

Solutions

1. Inspect ctx.errorLog() written by areAllRelevantIdsNonNull for which service failed
2. Check each clustered service's own logs for startup exceptions in onStart
3. Verify serviceCount and serviceId configuration match the deployed containers
4. Ensure service containers run the same Aeron version as the consensus module; restart the cluster

Example fix

// before: service crashing in onStart so its ack never arrives
public void onStart(Cluster cluster, Image snapshotImage) { throw new RuntimeException(...); }

// after: fail fast with a clear message and correct startup
public void onStart(Cluster cluster, Image snapshotImage)
{
    // validate dependencies before marking ready
    this.store = openStore(); // throws descriptive error if unavailable
}

// before starting the cluster, ensure all services are resolvable
for (ClusteredService service : services)
{
    Objects.requireNonNull(service, "null ClusteredService in configuration");
}

catch (AgentTerminationException e)
{
    if (e.getMessage().contains("failed to start clustered service"))
    {
        // consult ctx.errorLog() written by ServiceAck.areAllRelevantIdsNonNull
        log.error("a service never acked startup; check service container logs", e);
    }
    throw e;
}

When it happens

Trigger: Thrown when ServiceAck.areAllRelevantIdsNonNull fails after polling serviceAckQueues during cluster start, i.e. at least one service did not deliver its initial ack (ackId 0) in time.

Common situations: A service container crashed on startup (class not found, init exception); wrong service count or serviceId configuration; services blocked in their onStart; version mismatch between cluster and service containers.

Related errors

• initial ack already received from service: possible…
• not found in
• ${eventCode}: ${detail}
• clusterMembers and endpoints differ
• response publication already added

AI-assisted analysis of aeron-io/aeron@6d60124e15 (2026-09-12). Data as JSON: /api/errors/e63a5fb11ec52acc. Report an issue: GitHub.

Appendix: source

Thrown at aeron-cluster/src/main/java/io/aeron/cluster/ConsensusModuleAgent.java:3526

            if (!recoveryPlan.snapshots().isEmpty())
            {
                loadSnapshot(recoveryPlan.snapshots().get(0), archive);
            }
            else if (null != consensusModuleExtension)
            {
                consensusModuleExtension.onStart(this, null);
            }

            idleStrategy.reset();
            while (!ServiceAck.hasReached(expectedAckPosition, serviceAckId, serviceAckQueues))
            {
                idle(consensusModuleAdapter.poll());
            }

            final ServiceAck[] serviceAcks = ServiceAck.pollServiceAcks(serviceAckQueues);
            if (!ServiceAck.areAllRelevantIdsNonNull("failed to start clustered service", serviceAcks, ctx.errorLog()))
            {
                throw new AgentTerminationException("failed to start clustered service(s)");
            }

            captureServiceClientIds(serviceAcks);
            ++serviceAckId;
        }

        return recoveryPlan;
    }

    private RecordingLog.RecoveryPlan recoverFromBootstrapState()
    {
        final ConsensusModuleStateExport bootstrapState = ctx.bootstrapState();

        logRecordingId(bootstrapState.logRecordingId);
        final RecordingLog.RecoveryPlan recoveryPlan = recordingLog.createRecoveryPlan(
            archive, serviceCount, logRecordingId);

        expectedAckPosition = bootstrapState.expectedAckPosition;

View on GitHub (pinned to 6d60124e15)